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FOREWORD 


Concept  exploration  and  development  research  for  the  Army’s  transformation  to  Future 
Combat  Systems  (FCS)  is  a  key  concern  of  the  U.S.  Army  Research  Institute  for  the  Behavioral 
and  Social  Sciences  (ARI).  The  research  issues  and  approaches  identified  here  reflect  ongoing 
work  to  address  this  concern  by  the  Future  Battlefield  Conditions  (FBC)  Team  of  the  Armored 
Forces  Research  Unit  (AFRU).  This  report  supports  work  package  (211)  FUTURETRAIN: 
Techniques  and  Tools  for  Command,  Control,  Communications,  Computers,  Intelligence, 
Surveillance,  and  Reconnaissance  (C4ISR)  Training  of  Future  Brigade  Combat  Team 
Commanders  and  Staffs,  and  supports  the  Science  &  Technology  Objective  (STO)  '’''Methods  and 
Measures  of  Commander-Centric  Training." 

Creating  a  foreeful  alliance  of  humans  and  machines,  particularly  in  the  area  of 
Command  and  Control  (C^),  is  a  major  human-system  integration  challenge  for  FCS. 

Unintended  consequences  of  technology  include  an  increase  in  the  burden  on  humans  because 
human-system  integration  is  all  too  often  an  unattended  issue.  Moreover,  the  burden  on  humans 
from  advances  in  military  technology  is  caused  not  just  by  technology,  but  also  by  inflated 
expectations  about  technology. 

This  report  identifies  four  overarehing  research  issues  for  improving  human-system 
integration  in  the  area  of  command  and  control.  Then  two  complementary  research  approaches, 
mid-scale  and  small-seale  transformation  environments,  are  described  for  investigating  these  and 
related  human-system  integration  issues.  The  driving  and  integrative  focus  across  all  FCS 
transformation  efforts  should  be  the  need  to  shape  or  transform  technology  to  complement 
human  performance. 

The  mid-scale  transformation  environment  and  findings  reported  here  reflect  ARI’s 
ongoing  involvement  in  FCS  research,  including  FCS  C‘  program  efforts  to  explore  new 
paradigms  in  command  and  control.  The  emerging  small-scale  transformation  environment  at 
ARI-Fort  Knox  underscores  the  unique  and  eomplementary  roles  of  small-scale  research  with  a 
human-centric  focus.  These  roles  convey  research  in  smaller  environment  to  and  from  the 
Army’s  larger  FCS  transformation  environments. 

Results  of  this  effort  were  provided  to  the  Program  Manager  (PM)  FCS  C^  as  part  of 
ARI’s  ongoing  efforts  in  support  of  the  FCS  C^  research  program.  The  intended  audience  for 
this  report  includes  members  of  the  user,  researcher,  and  developer  eommunity  who  might 
benefit  from,  or  provide  benefit  to,  the  Army’s  ongoing  FCS  research  program. 


MICHAEL  G.  RUMSEY 
Acting  Technical  Director 
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HUMAN-SYSTEM  INTEGRATION  FOR  FUTURE  COMMAND  AND  CONTROL: 
IDENTIFYING  RESEARCH  ISSUES  AND  APPROACHES 

EXECUTIVE  SUMMARY _ 


Research  Requirement: 

The  research  requirement  addressed  in  this  report  reflects  the  unprecedented  alliance  of 
humans  and  machines  posed  by  Future  Combat  Systems  (FCS).  Achieving  that  alliance  is  a 
severe  human-system  integration  challenge  for  FCS,  particularly  for  Command  and  Control  (C^) 
at  the  small  unit  level.  This  report  identifies  research  issues  and  approaches  that  reflect  ongoing 
work  by  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  on  FCS 
transformation. 

Procedure: 

A  review  of  the  literature  selectively  identified  four  overarching  research  issues  for 
improving  human-system  integration  in  the  area  of  command  and  control:  Allocation, 
Autonomy,  Authority,  and  Awareness.  The  literature  review  and  ARI’s  experience  led  to 
several  key  conclusions.  First,  an  unintended  consequence  of  technology  is  that  it  often 
increases  the  burden  on  humans  because  human-system  integration  is  all  too  often  an  unattended 
issue.  Second,  the  burden  on  humans  from  advances  in  military  technology  is  caused  not  just  by 
technology,  but  also  by  inflated  expectations  about  technology.  Third,  the  Army  learns  by  doing. 

The  research  approaches  identified  here  underscore  the  need  for  leam-by-doing 
environments,  referred  to  as  transformation  environments.  A  mid-scale  transformation 
environment  is  identified  based  on  FCS  program  efforts  to  explore  new  paradigms  for 
command  and  control.  Description  of  this  environment  highlights  how  it  embodies  the  key 
features  of  a  transformation  environment:  empirical,  scalable,  iterative,  collaborative,  and 
human-centered.  Selected  results  from  FCS  C^’s  Experiment  1  are  provided  that  stress  human- 
system  integration  findings.  Lessons  learned  are  documented  for  refining  the  prototype  FCS  C^ 
environment  and  improving  human-system  integration  for  the  Unit  Cell  command  group. 

An  emerging  small-scale  transformation  environment  at  ARI-Fort  Knox  is  also  described. 
A  status  report  describes  this  environment’s  core  assets  for  creating  an  empirical  venue  that 
affords  users,  researchers  and  developers  the  ability  to  customize  tasks  and  conditions  in  order  to 
iteratively  explore  and  transform  concepts  into  viable,  human-centric  solutions.  Core  assets  are 
summarized  under  technical,  operational  and  human  performance  dimensions. 

Findings: 

The  report  concludes  that  a  decisive  value  added  by  small-scale  transformation  efforts  is 
the  ability  to  maintain  a  human-centric  focus,  often  lost  in  large-scale  efforts.  Moreover,  the 
ability  of  technology  to  situate  performance  provides  atypical  power  and  potential  to  small-scale 
transformation  environments.  Two  unique  and  complementary  roles  for  small-scale 


vn 


transformation  environments  are  identiffed  as:  a  breeding  ground  to  cull  and  refine  innovations 

for  transfer  to  larger  environments;  and,  a  proving  ground  to  assess  and  resolve  key  issues /row 
larger  environments. 

Utilization  of  Findings: 

Findings  from  this  effort  were  provided  to  the  Program  Manager  (PM)  FCS  as  part  of 
ARI’s  ongoing  efforts  in  support  of  the  FCS  research  program.  Lessons  learned  for  refining 
this  environment  and  improving  the  command  group’s  alliance  with  technology  should  help 
shape  future  FCS  research  and  acquisition  efforts.  Research  conducted  in  ARI’s  small-scale 
transformation  environment  will  focus  on  innovations  to  and  issues/row  the  Army’s  larger  FCS 
transformation  efforts,  including  Fort  Knox’s  role  as  proponent  for  FCS.  Findings  are  also  for 
members  of  the  user,  researcher,  and  developer  community  who  might  benefit  from,  or  provide 
benefit  to,  the  Army’s  ongoing  FCS  research  program. 
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HUMAN-SYSTEM  INTEGRATION  FOR  FUTURE  COMMAND  AND  CONTROL: 
IDENTIFYING  RESEARCH  ISSUES  AND  APPROACHES 


The  Army’s  transformation  to  Future  Combat  Systems  (FCS)  poses  an  unprecedented 
alliance  of  humans  and  machines.  Examples  of  the  alliance  include  humans  working  with 
intelligent  agents  or  “bots”  for  information  processing  and  decision  aiding,  and  with  robotic 
entities  for  moving,  seeing  and  shooting.  The  potential  impact  of  this  alliance  will  pervade  the 
force,  but  particularly  the  area  of  Command  and  Control  (C^)  where  expectations  about  new 
command  and  control  paradigms  are  emerging.  However,  creating  a  human-machine  alliance 
that  actually  improves,  and  does  not  impede,  command  and  control  is  a  human-system 
integration  challenge  for  FCS. 

This  report  is  a  small  part  of  a  large  and  expanding  body  of  government  and  private 
sector  work  on  human-system  integration.  An  impetus  for  much  of  this  work  is  the  growing 
requirement  for  humans  to  interact  with  highly  autonomous  and  complex  systems  that  are  all  too 
often  not  designed  to  support  such  interaction  (Olson  &  Sarter,  2001).  The  focus  of  this  report  is 
on  identifying  some  key  research  issues  and  approaches  directed  at  improving  human-system 
integration,  particularly  for  future  command  and  control  at  the  small  unit  level. 

This  report  selectively  identifies  four  overarching  research  issues  for  improving  human- 
system  integration  in  the  area  of  command  and  control:  Allocation,  Autonomy,  Authority,  and 
Awareness.  The  report  then  describes  two  complementary  research  approaches,  mid-scale  and 
small-scale  transformation  environments,  for  investigating  human-system  integration  issues. 

This  report’s  consideration  of  the  research  context  for  force  transformation  issues  and 
approaches  centers  on  several  key  themes.  The  profound  changes  essential  to  transformation 
require  a  comprehensive  research  approach  that  integrates  and  complements  efforts  across  a 
range  of  small  to  large  research  environments.  The  research  approaches  identified  here 
underscore  the  need  for  leam-by-doing  environments,  referred  to  as  transformation 
environments.  A  working  definition  of  a  transformation  environment  is  provided  and  its  key 
characteristics  are  identified. 

To  complement  FCS  research  efforts,  the  roles  of  small-scale  transformation 
environments  are  identified  as  a  breeding  ground  for  innovation  to  larger  environments,  and  a 
proving  ground  for  issu&s  from  larger  environments.  To  integrate  FCS  research  efforts,  the 
driving  focus  across  transformation  environments  and  efforts  should  be  the  need  to  shape  or 
transform  technology  to  complement  human  performance. 

The  research  issues  and  approaches  identified  here  reflect  preliminary,  ongoing  work  by 
the  U.S.  Army  Research  Institute’s  Future  Battlefield  Conditions  team  at  Fort  Knox.  A  primary 
purpose  of  this  documentation  is  to  disseminate  and  coordinate  our  team’s  efforts  within  the 
larger  body  of  current  and  planned  research  on  human-system  integration.  The  report’s  intended 
audience  includes  any  members  of  the  user,  researcher,  and  developer  community  who  might 
benefit  from,  or  provide  benefit  to,  this  evolving  research  program.  Acknowledgement  is  made 
to  the  many  members  of  that  audience  who’s  past  efforts  to  improve  the  interaction  of  humans 
and  machines  guide  future  efforts. 
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RESEARCH  ISSUES 


A  paradox  of  technology  is  that  advances  intended  to  ease  our  life  and  work  often  add 
complexity,  difficulty,  and  frustration  (Norman,  1988;  1997).  In  fact,  Norman  argues  that 
humans  and  modem  machines  are  becoming  increasingly  incompatible.  Humans  are  analog 
devices  designed  to  be  compliant,  flexible,  and  tolerant;  many  modem  machines,  particularly 
machines  based  on  the  exacting  nature  of  digital  technology,  require  us  to  be  rigid,  fixed,  and 
intolerant.  The  dilemmas  resulting  from  this  paradox  only  escalate  with  our  growing  reliance  on 
more  complex  and  automated  technology.  The  more  critical  our  reliance  on  technology  the 
more  pressing  the  need  for  research  directed  at  ensuring  technology’s  intended  consequences  and 
eliminating  its  unintended  consequences. 


This  paradox  is  of  particular  concern  to  the  design  and  development  of  PCS  that  entails 
m  extraordinary  amalgam  of  humans  and  machines,  a  tmly  hybrid  future  force.  Currently,  PCS 
IS  essentially  a  conceptual  design  featuring  an  interdependent  system-of-systems  (U.S.  Army 
Training  and  Doctrine  Command,  2001).  This  interdependence  is  reflected  in  the  concept  of  a 
netjvork-centric  force  composed  of  modular  manned  and  progressively  autonomous  platforms 
with  netted  communication,  sensor,  and  fire  capabilities. 

A  pivotal  example  of  the  human-system  integration  challenge  in  PCS  is  the  requirement 
that  a  relatively  small  command  group  can  command  and  control  an  expansive  mix  of  manned 
and  autonomous  systems.  This  command  group  requirement  for  the  Unit  Cell,  the  smallest 
combined  arms  echelon  within  the  PCS  structure,  is  the  immediate  focus  of  the  research 
discussed  here.  This  challenge  is  compounded  by  the  command  group’s  unprecedented  reliance 
on  tectoology,  and  the  expectation  that  technology  will  enable  new  paradigms  in  command  and 
control  (Wass  de  Czege,  in  preparation).  In  sum,  for  PCS  the  integration  of  humans  and 
automation  is  integral  to  conceptualizing  and  performing  command  and  control. 


The  research  issues  confronting  PCS  human-system  integration  are  numerous  and 
daunting.  Many  of  those  issues  are  common  to  multiple-disciplines  within  and  beyond  the 
behavioral  sciences.  “In  principle,  it  would  be  difficult  to  find  a  theory,  paradigm,  or  body  of 
empirical  results  in  the  cognitive  and  psychological  sciences  without  the  potential  to  illuminate 
some  issue  or  problem  in  human-machine  interaction”  (Kirlik  &  Bisantz,  1999,  p.  48).  Notably 
this  report  makes  no  attempt  to  review  or  summarize  the  vast  and  growing  literature  on  human- ' 
SdvT^d”(T997°”  reviews,  interested  readers  are  referred  to  Hancock  (1999)  and 


The  goal  of  this  section  is  to  selectively  identify  a  smaller,  more  workable  set  of 
overarching  research  issues  for  improving  human-system  integration  in  the  area  of  future 
command  and  control.  The  issues  identified  were  selected  based  on  their  perceived  relevance  to 
future  command  and  control.  Table  1  summarizes  the  four  research  issues  identified  that  are 
bnefly  discussed  below. 
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Table  1 


Key  Human-System  Integration  Issues  Identified  for  Future  Command  and  Control 


Allocation 

Interdependent  tactics  and  technology  constantly 
stressed  the  force  to  the  limits  of  human  capability.* 

How  to  allocate  human- 
machine  flmctions? 

Authority 

Automation  limits  and  overrides  human  operators."' 

Who  is  in  command? 

Autonomy 

Automated  systems  change  their  own  behavior  in 
response  to  changes  in  the  situation.^ 

Who  is  in  control? 

Awareness 

Commanders  with  digital  systems  had  problems 
maintaining  the  big  picture.^ 

How  to  maintain 
the  big  picture? 

'  Cordesman  &  Wagner  (1996)  Sarter,  &  Woods  (2000)  ^McGuiness,  Foy  and  Forsey  (2000) 


Allocation 

How  to  best  allocate  human-machine  functions?  The  issue  of  how  to  allocate  tasks  and 
functions  is  a  growing  concern  for  force  transformation.  This  discussion  first  highlights  the  issue 
of  human-machine  allocation  in  military  operations  in  general,  and  then  focuses  directly  on  the 
area  of  command  and  control. 

A  fundamental  lesson  from  modem  warfare  is  that  the  insertion  of  technology  tends  to 
burden  and  stress  the  force,  as  documented  by  Cordesman  &  Wagner  (1996).  Their  analysis  of 
the  Gulf  War  concluded  that  the  war’s  influx  of  new  technology  stressed  the  force  to  the  limits  of 
human  capability.  Many  of  the  technologies  introduced  were  expected  to  ease  operator  burden, 
reduce  fatigue,  and  simplify  combat.  Rather  than  unburdening  military  persormel,  however, 
high-tech  operations  substantially  raised  the  bar  on  human  expertise,  commitment,  and 
endurance. 

Notably,  the  burden  on  humans  associated  with  advances  in  military  technology  is 
attributed  less  to  technology  per  se,  than  to  inflated  expectations  about  technology.  Cordesman 
and  Wagner  (1996)  stress  that  the  impact  of  heightened  expectations  on  humans  is  pervasive; 
including  technology  enables  more  sorties,  faster  maneuver,  extended  fires,  just-in-time  logistics, 
and  continuous  operations.  Moreover,  “more  with  less”  expectations  about  technology  almost 
always  include  a  “more  with  fewer  personnel”  premise. 

The  issue  of  how  best  to  allocate  human-machine  functions  and  tasks  is  especially 
problematic  in  the  area  of  command  and  control.  Although  automated  systems  are  beginning  to 
relieve  commanders  and  staffs  of  many  repetitive  and  computational  tasks,  the  human  tasks  that 
remain  are  the  most  challenging  and  critical  (Taylor,  Charlton  &  Canham,  1996,  p.  301).  Many 
command  and  control  tasks  are  too  complicated  and  too  important  to  assign  to  machines.  For 
battle  command,  for  example,  advanced  technologies  should  help  commanders  visualize  the 
operation,  describe  it  within  their  intent,  and  direct  subordinates  toward  accomplishing  the 
mission.  However,  the  “science”  of  technology,  severely  lags  the  “art”  of  battle  command. 
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Authority 


Who  IS  in  command?  The  issue  of  command  is  a  fundamental  concern  as  technology 
becomes  more  autonomous.  The  high  degree  of  automation  occurring  in  some  operational 

overrides  human  decision  makers,  such  as  aircraft  pilots  (Sarter 
&  Woods,  2000).  Commercial  aviahon  serves  as  a  leading,  natural  laboratory  for  investigating 

uman-system  integration  issues  as  it  pioneered  the  introduction  of  autonomous  systems  in  high- 
nsk  operational  settings.  ^ 

^  ^  authority  in  aviation  is  “envelope  protection” 

(S^er  &  Woods,  2000).  Envelope  protection  is  the  power  of  air  systems  to  automatically 

rnnra  ^  of  predefined  situational  conditions  occurs,  namely  unsafe  air 

configurations.  \^en  the  aircraft  approaches  a  defined  unsafe  condition,  automation  assumes 

direct  confrol  of  the  aircraft  to  avoid  or  recover  from  that  condition,  and  this  control  includes  the 
power  to  limit  or  override  pilot  input. 

As  the  projected  capabilities  of  PCS  systems  evolve,  issues  of  when  and  if  to  keep  the 
human  in  charge  will  become  increasingly  prominent.  The  system-of-systems  architecture  of 
PCS  will  raise  system  complexity  and  lower  system  observability  to  where  informed  command 
and  even  consent  over  autonomous  systems  is  lost  or  imperiled.  A  revealing  example  is  how  the 
anticipated  introduction  of  higher-order  automation  such  as  “intelligent”  agents  oftL  adds 
complexity  to,  and  reduces  observability  of,  system  interactions.  Intelligent  agents  are  software 
routines  or  programs  that  work  as  mediators  between  humans  and  lower  level  technology. 

Por  PCS  cor^and  and  control,  intelligent  agents  are  expected  to  perform  many  roles 
such  as  aiding  decision-making  and  “controlling”  the  execution  of  decisions  as  carried  out  by 
unm^ed  sensor  and  weapon  platforms.  Intelligent  agents  are  expected  to  “understand”  the 
directives  and  requests  of  humans,  and  then  adequately  translate  these  human  inputs  into 
software  commands”  to  lower-level  automated  systems.  However,  the  complexity  and 

induced  by  mediating  layers  of  technology  can  severely  restrict  the  ability  of  humans 
(ofsoT&‘saX,?00ir^"'  difficulties  and  consequences,  or  even  to  provide  informed  consent 


Autonomy 

f  control?  The  issue  of  control  is  proving  to  be  a  problematic  issue  with  more 
autonomous  technology.  The  authority  of  command  is  often  extended  into  control  through  the 
formulation  of  standing  operating  procedures  (SOPs)  that  predefine  and  thereby  limit  the  manner 
in  which  explicit  and  implicit  commands  must  be  performed.  The  autonomous  capabilities  of 

technology  can  be  sirnil^ly  curtailed  by  encoded  procedures,  software  routines  that  restrict  the 
potential  autonomy  of  advanced  technology. 

Controlling  strategies  for  managing  automation  such  as  management-by-exception  and 
management-by-consent  are  receiving  increased  scrutiny  (Olson  &  Sarter,  2001 )  In  the  case  of 
managernent-by-exception,  machines  are  allowed  to  initiate  and  perform  actions  independently 
This  strategy  reduces  human-machine  interaction  by  the  operator,  but  often  demands  more 
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intense  human  monitoring  and  a  loss  of  awareness  about  the  system  and  the  situation. 
Conversely,  a  management-by-consent  strategy  requires  the  automated  system  to  ask  for  and 
receive  human  permission  before  acting.  A  consent  strategy  tends  to  increase  human  awareness 
about  the  system  and  the  situation,  but  also  increases  human  interaction  demands. 

Operational  answers  about  who  is,  or  who  should  be,  in  control  are  neither  clear  nor 
fixed.  As  might  be  expected,  humans  in  responsible  roles  initially  opt  for  a  management-by¬ 
consent  strategy  in  order  to  maintain  control  (Olson  &  Sarter,  2001).  However,  these  same 
humans  often  shift  their  preferred  strategy  on  many  tasks  to  management-by-exception  after 
experiencing  high  workload,  time  pressure,  and  complex  task  requirements. 

Preferences  aside,  objective  performance  data  indicates  that  management-by-consent 
does  not  ensure  effective  control.  Olson  and  Sarter  (2001)  suggest  that  despite  its  appeal,  a 
management-by-consent  strategy  is  severely  compromised  by  technology  advances  that  result  in 
increasingly  complicated  and  obscure  automated  systems.  However,  management-by-exception 
strategies  entail  severe  risk,  greater  reliance  on  automation,  and  a  decrease  in  human  control  and 
understanding  of  the  system  and  situation. 


Awareness 

How  to  maintain  human  awareness  of  “the  big  picture?”  The  issue  of  human  awareness 
is  emerging  as  a  pervasive,  and  to  some  degree  unexpected,  human-system  integration  issue  the 
more  technology  advances.  The  ability  of  advanced  information  teclmology  to  increase  the 
situational  awareness  of  commanders  and  combatants  by  maintaining  a  common  picture  of  the 
tactical  situation  is  a  basic  expectation  of  PCS.  Clearly,  technology  is  steadily  improving  the 
ability  to  provide  more  accurate  and  timely  depictions  of  tactical  situations  to  commanders, 
combatants  and  supporters  in  mounted  and  dismoimted  settings.  However,  the  ability  to  provide 
more  data  and  information  often  limits  and  distorts  human  awareness. 

A  particular  concern  for  command  and  control  is  the  potential  of  information  technology 
to  limit  commanders’  overall  awareness  of  the  tactical  situation.  For  example,  McGuinness,  Foy 
and  Forsey  (2000)  compared  the  ability  of  Battlegroup  commanders,  equipped  with  digital 
systems  versus  voice  radio  and  paper  maps,  to  plan  and  execute  a  land  reconnaissance  operation. 
The  only  difference  detected,  across  measures  of  workload  and  situational  awareness,  was  that 
commanders  with  systems  reported  a  significant  decrease  in  their  ability  to  comprehend  the 
overall  tactical  situation. 

A  similar  concern  is  emphasized  by  aneedotal  assessments  on  the  impact  of  live  video 
from  semi-autonomous  Predator  drones  flying  in  Afghanistan  (Ricks,  2002).  A  brigade 
commander  reported  he  hardly  watched  the  video  because  “the  Predator  can  be  mesmerizing  — 
like  watching  TV.”  A  division  fire  support  coordinator  stated:  “The  danger  is  you  get  too 
focused  on  what  you  can  see,  and  neglect  what  you  can't  see.”  Additional  concerns  about  the 
impact  of  this  unmanned  aerial  vehicle  (UAV)  included  assessments  that  beaming  the  video  to 
higher  levels  of  command  led  to  micromanagement  of  operations. 
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Advanced  technology  impacts  human  awareness  at  many  levels,  not  just  the  ability  to 
maintain  an  overall  awareness  of  the  situation,  the  bigger  picture.  In  particular,  the  issue  of 
maintaining  system  awareness  with  more  autonomous  systems  has  resulted  in  substantial 
research  and  findings  that  should  be  extended  to  PCS.  Problematic  issues  in  system  awareness 
such  as  mode  awareness  and  automation  surprises,  reflect  the  difficulty  humans  experience  in  ’ 
momtonng,  anticipating,  and  redirecting  more  autonomous  systems  (e.g.,  Sarter,  Billings  & 


BACKGROUND  ON  RESEARCH  APPROACHES 

u  transformation  to  Future  Combat  Systems  (PCS)  entails  profound 

chmge.  The  PCS  concept  requires  a  readily  adaptive  system-of-systems  design  to  counter 
unforeseeable  changes  in  threat  capabilities  and  operations,  and  enable  new  paradigms  for 
command  and  control.  As  underscored  iii  this  report,  the  PCS  concept  also  requires  an 
unprecedented  alliance  of  humans  and  increasingly  autonomous  machines.  Clearly,  there  is 
much  to  be  learned  to  transform  PCS  concepts  into  reality.  The  Army  learns  by  doing. 

Transformation  Environments 


To  leam  by  doing  requires  an  empirical  environment  structured  to  support  key  learning 
Objectives,  including  exploratory  learning  objectives.  The  research  approaches  identified  in  tWs 
section  include  the  development  of  the  required  leam-by-doing  environment,  referred  to  as  a 
transformation  environment.  A  working  definition  of  a  transformation  environment  is  provided 
to  guide  efforts  m  developing  mid-  and  small-scale  transformation  environments  for  FCS 
research  directed  at  human-system  integration. 


Transformation  environments  are  empirical  venues  that  afford  users,  researchers  and 
developers  the  ability  to  customize  tasks  and  conditions  in  order  to  iteratively  explore  and 
transform  concepts  into  viable,  human-centric  solutions.  Key  features  that  distinguish  a 
transformation  environment  are  bulleted  below. 


•  Empirical — ensure  performance-based  human-system  interaction. 

•  Scalable— customize  tasks  and  conditions  to  research  issues  and  objectives. 

•  Iterative — focus  on  performance  refinement. 

•  Collaborative— join  users,  researchers  and  developers. 

•  Human  shape  technology  to  complement  human  performance. 

More  complete  descriptions  and  examples  of  these  features  are  provided  in  the  following 
Research  Approaches  section  that  describes  ongoing  development  of  mid-  and  small-scale 
transformation  environments  directed  at  FCS  command  and  control. 

Notably,  environments  directed  at  transformation  and  concept  exploration  are  not  new 
and  are  increasingly  common  in  government  and  private  sectors  (Gold,  1999)  Moreover  the  set 
of  environment  features  listed  above  are  certainly  not  new  and  are  common,  at  least  in  pak  to 
many  research  and  development  efforts.  In  particular,  the  scale  of  a  research  environment  is 
central  to  classic  distinctions  between  field  and  laboratory  research  and  their  respective  ability  to 


6 


provide  externally  and  internally  valid  results  and  findings.  Similarly,  the  need  for  an  iterative 
focus  on  concept  and  performance  refinement  is  underscored  in  the  “spiral  development”  process 
pioneered  by  the  Army  for  force  transformation  (Gold,  1999). 

However,  the  profound  changes  entailed  in  transformation  require  a  unique  mix  of 
environmental  features.  The  set  of  features  proposed  above  addresses  the  sweeping  purpose  and 
scope  of  a  transformation  effort.  In  contrast,  severe  shortcomings  often  exist  in  the  environments 
designed  or  cobbled  together  to  foster  transformation.  For  force  transformation,  multiple 
environments  directed  at  different  levels  and  issues  are  required,  and  their  efforts  must  be  closely 
coordinated. 

Human  considerations  are  often  lost  in  large-scale  efforts.  Moreover,  leam-by-doing 
transformation  environments  require  a  re-configurable  performance-based  venue  that  can 
customize  tasks  and  conditions  to  learning  objectives.  The  emphasis  on  an  empirical 
environment  underscores  the  need  for  objective  measures  of  performance  based  on  human- 
system  interaction  with  prototype  or  actual  FCS  equipment.  The  ability  to  shape  or  scale  the 
environment  is  crucial  to  iteratively  refining  concepts  through  lessons  learned  empirically. 
Notably,  simulation  environments  are  inherently  scalable,  and  virtual  simulation  affords  user-in- 
the-loop  assessment  and  feedback.  But  larger  scale  environments,  even  when  simulated,  are 
more  difficult  and  costly  to  reconfigure  than  mid-  and  small-scale  environments. 

In  sum,  developing  the  interdependent  system-of-systems  envisioned  by  FCS  will  require 
a  wide  range  of  complementary  and  closely  integrated  transformation  environments.  Building 
and  sustaining  an  FCS  infi-astructure  of  transformation  environments  is  essential  to  building  and 
sustaining  a  rapidly  adaptive  force.  Force  dominance  based  on  adaptability  depends  not  so  much 
on  technology,  but  upon  the  ability  of  this  infrastructure  to  integrate  technology  into  the  force 
(Carter  &  White,  2000).  This  ability  equates  to  human-system  integration,  the  requirement  to 
shape  technology  to  complement  human  performance. 

A  Scaled  Research  Approach  Example 

An  example  of  a  scaled-down  research  environment  is  described  that  stresses  the 
importance  of  being  able  to  customize  tasks  and  conditions  to  meet  research  issues  and 
objectives.  The  ability  to  scale  down  a  research  environment  for  a  more  human-centric  focus  is 
of  particular  concern  to  the  research  approaches  described  in  this  report.  The  theme  of  research 
scale  is  also  used  to  relate  the  smaller  scale  research  approaches  described  in  this  report  to  more 
comprehensive  efforts,  including  large-scale  transformation  environments. 

An  example  of  why  and  how  to  scale  down  a  research  approach  in  support  of  command 
and  control  transformation  is  provided  in  the  scaled  world  approach  developed  by  Ehret,  Gray 
and  Kirschenbaum  (2000).  This  example  also  reflects  the  exploratory  mindset  required  for 
profound  transformations:  “We  believe  that  the  only  aspects  of  the  submarine  officers’  job  that 
would  not  change  in  the  next  25  years  are  the  laws  of  physics  regulating  underwater  sound 
propagation”  (Ehret  et  al.,  2000,  p.l3). 
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^  answer  as  to  why  a  scaled-down  research  approach  is  needed  is  provided  in  their 
description  of  a  scaled  world.  A  scaled  world  research  approach  serves  as  a  middle  ground 
between  the  situational  complexity  inherent  in  field  research  that  resists  definite  conclusions  and 
the  situational  paucity  of  laboratory  research  that  defies  useful  conclusions  (Ehret  et  al  2000) 

By  design,  it  preserves  key  functional  relationships  based  on  questions  of  interest  to  evaluators 
or  trainers,  while  paring  away  other  functions  that  might  confound  answering  those  questions. 

A  scaled  world  approach  was  adopted  to  address  the  recurrent  problem  of  tracking  and 
^  oAAm  humans  process  information  as  they  perform  command  and  control  tasks  (Ehret  et 
al.,  2000).  The  goal  was  to  derive  a  step-by-step  record  of  how  officers  process  information  on 
their  C  systems  to  localize  an  enemy  submarine  hiding  in  deep  water.  A  scaled  world  approach 
was  used  to  overcome  problems  encountered  by  the  researchers  in  a  high-fidelity  environment 
Problems  in  tracking  officers’  information  processing  as  they  used  high-fidelity  systems 
included:  redundant  information  across  displays;  un-requested  information  encountered  while 
navigating  through  intermediate  displays,  and  copious  information  within  displays  that 
encouraged  browsing. 

To  overcome  such  problems,  a  scaled-world  version  of  the  system  was  developed  to 

more  precisely  track  the  information  accessed  by  officers  to  localize  enemy  submarines  Scale- 
down  modifications  to  the  C  system  included  paring  away  some  auxiliary  functions,  such  as 
eliminating  redundant  information  across  displays  and  navigational  changes  that  eliminated 
intermediate  displays.  At  the  same  time,  these  modifications  carefully  ensured  that  the  basic 
functions  of  the  C  system  were  retained,  including  all  information  available  for  the  localization 


Asa  result  of  this  scaled  world  approach,  the  researchers  were  able  to  identify  and 
subsequently  model  officers’  information  processing  activities  (350-450  information  accesses 
per  scenario).  The  model  was  derived  from  logs  of  human-system  interaction  that  were 
automatically  segmented  into  officer-defined  goals  with  95  percent  accuracy.  Empirical 
compansons  between  the  officers’  scaled  world  and  real  world  performance  were  strongly 

supported,  and  the  officer-expert  model  derived  from  this  approach  provides  an  excellent  basis 
for  traimng  (Ehret  et  al.,  2000). 


Large-Scale  Transformation  Environments 


Before  turning  to  the  research  approaches  central  to  this  report— mid-scale  and  small- 
scale  transformation  environment— a  very  brief  discussion  of  large-scale  environments  in 
support  of  transformation  is  provided.  The  intent  of  this  discussion  is  to  reinforce  the  need  for 
complementary  research  approaches  to  address  PCS  human-system  integration  issues. 

A  cogent  assessment  of  force  transformation  across  the  services  by  the  Defense  Science 
Board  focused  on  the  requirements  for,  and  shortcomings  of,  large-scale  efforts  (Gold  1999) 
The  Board’s  report  begins  with  the  understanding  that  transformation  is  a  process  that’seeks  ’ 
fundamental  change  in  how  an  enterprise  conducts  its  business.  Their  report  notes  that  the 
transformation  initiatives  across  the  services  seek  common  changes  including:  rapid 
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deployment,  agility,  small  footprint,  reachback,  distributed  operations,  and  exploitation  of 
information  in  all  dimensions  of  operation  and  planning. 

Tbe  Board’s  requirements  for  large-scale  transformation  efforts  stress  tbe  need  for  cross¬ 
service  concept  development  and  experimentation.  To  address  this  need,  tbe  Board  urged 
decision  makers  to  exercise  tbe  entire  transformation  process — from  concept  development 
through  implementation — as  soon  as  possible  to  address  tough  issues  early  on.  To  manage 
large-scale  efforts,  the  Board  recommended  that  objectives  be  limited  to  a  manageable  set.  To 
facilitate  that  recommendation,  they  suggested  transformation  should  not  focus  on  all  force 
elements,  but  on  the  synergies  created  by  new  and  legacy  forces.  Notably,  they  also  stressed  that 
an  urgent  and  steadfast  commitment  to  transformation  by  a  few,  is  crucial  to  overcoming  the 
inertia  and  resistance  of  many. 

However,  many  shortcomings  in  large-scale  transformation  efforts  were  cited  in  the 
Board’s  report.  In  particular,  transformation  efforts  must  “...avoid  focusing  on  so-called  ‘Super- 
BowT  events — expensive  and  unwieldy  activities  that  inhibit  learning”  (Gold,  1999,  p.  26). 
Instead,  the  report  recommended  that  the  transformation  process  employ  a  broad-based  iterative 
approach  using  a  variety  of  tools  and  venues  that  can  lead  to  early  discovery. 

Many  shortcomings  inherent  to  large-scale  environments,  including  transformation, 
training  and  testing  efforts,  are  common  to  both  field  and  simulation  settings.  An  analysis  of 
large-scale  simulations  by  Simpson  (1999)  also  concluded  they  usually  focus  on  events  versus 
process.  Inevitably,  the  high  costs  in  human  and  system  resources  required  for  large-scale  efforts 
prohibit  an  iterative  process  of  exploration  or  evaluation  geared  toward  improvement. 

For  command  and  control  transformation,  limitations  in  large-scale  environments  are 
particularly  severe.  In  part,  the  sheer  size  and  complexity  of  modem  systems  complicates 

assessment  and  often  limits  its  focus  to  technical  versus  human  dimensions  (Taylor  et  al.,  1996). 
In  addition,  technology  advances  such  as  software  upgrades  to  systems  occur  so  frequently 
that  iterative  assessment  is  required.  Taylor  et  al.  (1996)  also  conclude  that  more  comprehensive 
research  approaches  are  needed  for  command  and  control  that  include  an  integrated  mix  of 
demonstrations,  experiments,  simulations,  exercises,  and  formal  tests. 

In  sum,  large-scale  transformation  efforts  are  an  essential  component  of  research  and 
development  efforts  directed  at  transformation,  particularly  in  terms  of  external  validity — ^value 
to  the  force.  However,  the  focus  of  this  report  now  returns  to  the  need  for  complementary 
research  approaches  based  on  mid-  and  small-scale  transformation  environments. 

RESEARCH  APPROACHES 

The  research  approaches  identified  here  underscore  the  need  for  leam-by-doing 
transformation  environments.  A  mid-scale  transformation  environment  is  identified  based  on 
FCS  C^  program  efforts  to  explore  new  paradigms  for  command  and  control.  Description  of  this 
environment  highlights  how  it  embodies  the  key  features  of  a  transformation  environment: 
empirical,  scalable,  iterative,  collaborative,  and  human-centered.  Selected  results  from  this 
effort’s  Experiment  1  are  provided  that  stress  human-system  integration  findings.  Lessons 
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learned  are  documented  for  refining  this  environment  and  improving  human-system  integration 
for  the  Unit  Cell  command  group. 


An  emerging  small-scale  transformation  environment  at  ARI-Fort  Knox  is  also  described. 
A  status  report  describes  this  environment’s  core  assets  for  creating  an  empirical  venue  that 
affords  users,  researchers  and  developers  the  ability  to  customize  tasks  and  conditions  in  order  to 
Iteratively  explore  and  transform  concepts  into  viable,  human-centric  solutions.  Core  assets  are 
summarized  under  technical,  operational  and  human  performance  dimensions. 

Mid-Scale  Transformation  Environments 

This  section  describes  an  emerging  mid-scale  transformation  environment  in  support  of 
PCS  command  and  control.  As  noted,  this  research  approach  reflects  preliminary  and  ongoing 
^  conjunction  with  other  research  and  development  teams.  This  brief  description 
will  highlight  how  this  environment  reflects  many  of  the  key  features  of  a  transformation 
environment  previously  considered.  As  research  progresses  beyond  the  Experiment  1  effort 
summarized  here,  these  environmental  features  will  become  more  robust. 

A  case  example  of  a  mid-scale  transformation  environment  for  exploring  command  and 
confrol  concepts  at  the  small  unit  level  is  the  PCS  C^  program.  This  program  is  a  joint  effort  led 
by  Defense  Advanced  Research  Projects  Agency  (DARPA)  and  U.S.  Army  Communications- 
Electromcs  Command  (CECOM)  Research  and  Development  Center  (RDEC).  During  this 
program,  October  2001  to  March  2003,  an  iterative  series  of  command-in-the-loop  experiments 
will  be  conducted  at  CECOM.  As  a  participating  member  in  this  effort,  the  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  serves  primarily  on  the  PCS  C^  Human 
Performance  Team  and  performs  multiple  research  roles: 

•  Advises  on  experimental  design,  measurement,  and  training  issues. 

•  Analyses  command  group  behaviors  through  observation  and  post  hoc  analysis  of 
recorded  experimental  trials. 

•  Conducts  complementary  in-house  research  directed  at  improving  human-system 
integration  in  the  area  of  command  and  control  (see  Small-Scale  Transformation 
Environments  section  that  follows). 

The  FCS  Transformation  Environment 

The  stated  purpose  of  the  FCS  C^  program  is  to  test  the  hypothesis  that  digitization  of 
current  battlefield  operating  systems  enables  a  new  approach  to  command  and  control: 

If  digitization  of  current  battlefield  operating  systems  can  substantially  enhance 
command  and  control  by  providing  better,  more  accurate,  and  timely  battlefield  data  to  today ’s 
commander  and  staff  for  decision  making;  then  a  ‘new  ’  approach  to  Battle  Command  and 
Control,  implemented  in  the  form  of synthesized/analyzed  information  presented  to  the  future 
Unit  Cell  Commander,  will  enable  him  to  leverage  of  opportunities  by  focusing  on  fewer 
unknowns,  clearly  visualizing  current  and  future  end  states,  and  dictating  the  tempo  within  a 
variety  of  environments,  while  being  supported  by  a  significantly  reduced  staff.  " 
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To  test  this  hypothesis,  the  PCS  C  program  created  and  continues  to  refine  a  prototype 
transformation  environment  for  empirical  assessment  of  command  group  performance  at  the  Unit 
Cell  level.  The  concept  of  the  PCS  Unit  Cell  is  evolving  including  its  size  and  capability. 
Currently,  the  Unit  Cell  is  the  smallest  combined  arms  echelon  within  the  PCS  structure.  The 
Unit  Cell  concept  proposes  that  a  small  command  group — a  commander  and  1-5  additional 
personnel — can  command  and  control  a  substantial  number  of  manned  and  robotic  elements 
performing  a  wide  range  of  battlefield  functions  including  recoimaissance,  surveillance, 
targeting,  and  acquisition.  Pigure  1  depicts  the  maimed  and  robotic  elements  of  the  Unit  Cell 
employed  in  Experiment  1,  including  the  C^  Vehicle  occupied  by  the  cell’s  command  group. 


Unit  Cell’s  Organic  Assets 


2  Line  of  Sight  (tOS)  Weapons 


2  Robotic  Scout  Vehicles 


2  Future  Warrior  Carriers 


2  Nino  Man  Dismoun  t  Squads* 


1  FCS  C2  Vehicle 


^  4  Unmanned  Aerial  Vehicles  (UAVs) 

H  N 


2  NonLOS/Beyond  Line  of  Sight  Weapons 


Pigure  1.  Organization  of  the  Unit  Cell  during  Experiment  1. 

The  resources  and  products  of  three  interdependent  teams — Operational,  Technical,  and 
Human  Performance — were  required  to  create  this  transformation  environment  for  Experiment  1. 
The  Technical  Team  developed  the  Commander’s  Support  Environment  (CSE),  a  hardware  and 
software  system,  located  in  the  command  group’s  C^  Vehicle.  The  CSE  workstations  for  each 
member  of  the  command  group — Commander,  Battle  Space  Manager,  Information  Manager,  and 
Effects  Manager — allowed  them  to  command  and  control  their  Unit  Cell  elements. 

The  Technical  Team  also  developed  support  technologies  such  as  the  Collaborative 
Server  so  the  command  group  could  share  information  via  a  common  operational  picture,  and  the 
Collective  Intelligence  module  to  ensure  the  Unit  Cell’s  elements  worked  together  in  a  network 
centric  environment.  Through  the  CSE’s  links  to  Distributed  Interactive  Simulation  (DIS),  the 
command  group  interacted  with  simulated  elements  of  the  Unit  Cell,  the  threat  force,  and  civilian 
entities. 

The  Operational  Team  collaborated  on  CSE  design,  developed  the  mission  requirements 
and  scenarios  for  Unit  Cell  operations,  and  provided  players  and  supporters  for  Experiment  1. 

The  command  group  players  were  four  active  duty  U.S.  Army  Lieutenant  Colonels  deliberately 
selected  to  help  explore  and  develop  new  paradigms  for  command  and  control.  These  players 
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were  ably  complemented  by  the  expertise  and  vision  provided  by  supporting  personnel  serving 
as  Friendly  and  Enemy  commanders  and  an  Observer/Controller  (0/C)  team. 

The  Human  Performance  Team  devised  and  implemented  training  and  evaluation 
methods  compatible  with  an  incremental  series  of  experiments  designed  to  explore  and  document 
lessons  learned  for  Army  transformation  and  acquisition  objectives.  Team  efforts  focused  on  the 
human-machine  interactions  required  for  the  Unit  Cell  concept,  and  stressed  that  a  forceful 
alliance  required  shaping  technology  to  complement  human  performance.  This  team  collected 
and  analyzed  a  variety  of  human  performance  data  during  Experiment  1,  as  discussed  in  a 
following  section  titled  Interim  Findings:  A  Focus  on  Human-System  Integration. 

Notably,  formation  and  sustainment  of  an  environment  for  concept  exploration  and 
development  is  required  to  transform  FCS  concepts  into  viable  solutions.  Experiment  1  assessed 
only  the  ability  of  the  Unit  Cell  to  move  its  elements  in  order  to  see  the  enemy  and  not  be  seen 

(i.e.,  See/Move).  By  Experiment  4,  Unit  Cell  missions  will  include  Improved  See/Move/Strike/ 
Sustain  and  Transition  requirements. 

•  Experiment  1— Dec.  2001— See/Move. 

•  Experiment  2— May  2002— Improved  See/Move  and  Strike. 

•  Experiment  3  Sep.  2002 — Improved  See/Move/Strike  and  Sustain. 

•  Experiment  4  Feb.  2003  Improved  See/Move/Strike/Sustain  and  Transition. 

technologies  developed  for  Experiment  1  represented  only  about  20  percent 
of  the  full  functionality  envisioned  for  the  Unit  Cell’s  command  group.  As  the  experiments 

progress,  new  technologies  will  be  added,  and  current  technologies  will  be  refined  or  abandoned 
based  on  lessons  learned. 


Experiment  I  Overview 

The  FCS  C^  Experiment  1  was  conducted  from  3-14  December,  2001.  During  the  first 
week,  program  personnel  trained  the  four  command  group  players  on  operation  of  the  CSE. 
During  the  second  week,  the  actual  experiment  was  conducted.  A  total  of  nine  (9)  experimental 
trials  were  run  based  on  the  Unit  Cell’s  See/Move  mission.  After  selected  trials,  the  0/C  team 
led  After  Action  Reviews  (AAR)  that  addressed  operational,  technical,  and  human  performance 


The  efforts  of  ARI  in  support  of  training  and  evaluation  resulted  in  the  use  of  deliberate 
practice  methods,  and  the  manipulation  of  trial  complexity.  Experiment  1  required  that  the 
players  plan  and  execute  essentially  the  same  See/Move  exercise  across  all  experimental  trials 
The  deliberate  practice  design  included  AAR  performance  feedback  and  afforded  the  players  an 
opportunity  to  learn  a  demanding  set  of  new  command  and  control  skills.  The  design  also 
allowed  experimenters  to  vary  trial  conditions  as  a  function  of  METT-TC  (mission,  enemy, 
terrain,  troops,  time  and  civilians)  among  “Medium,”  “High”  and  “Too  High”  levels  of  trial 
complexity  in  order  to  gauge  the  performance  limits  of  the  Unit  Cell. 
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Interim  Findings:  A  Focus  on  Human-system  Integration 

Results  from  Experiment  1  are  interim  findings.  These  findings  serve  as  benchmarks  for 
subsequent  experiments  as  well  as  formative  development  of  the  CSE  and  new  paradigms  of 
command  and  control.  Results  include  objective  and  subjective  measures  of  effectiveness  and 
performance  by  the  command  group,  the  Unit  Cell,  and  the  CSE.  For  example,  one  key  measure 
for  the  experiment’s  focus  on  See/Move  was  the  percentage  of  threat  elements  located  by  the 
Unit  Cell  during  a  trial  as  a  function  of  trial  complexity  (Medium,  High,  and  Too  High). 

Detailed  results  are  documented  in  the  Interim  Report  for  Experiment  1,  available  from  the 
Program  Manager  (PM)  FCS  C^. 

Interim  findings  on  changes  needed  to  improve  human-system  integration  for  the  Unit 
Cell’s  command  group  were  also  documented  (Sanders  et.  al.,  2002).  The  data  for  these  findings 
were  collected  from  questionnaires,  structured  interviews,  and  fully  recorded  trials  and  AARs. 
These  results  include  numerous  recommendations  from  the  command  group  players  and  AAR 
facilitators  for  improving  CSE  and  command  group  performance. 

Researchers  from  ARI  observed  Experiment  1  training  and  trials,  and  administered  three 
data  collection  instruments:  In-Place  AAR,  After  Exercise  Survey,  and  Exit  Interview.  In 
addition,  ARI  researchers  performed  a  post  hoc  analysis  of  the  verbal  communications  by  the 
command  group  and  higher  headquarters  cells  during  the  experiment’s  nine  runs  in  support  of  a 
Human  Fimctions  Assessment.  Next,  a  small  sample  of  results  from  each  of  these  four  research 
activities  is  presented.  For  interested  readers,  detailed  documentation  on  the  method,  results  and 
conclusions  for  this  effort  are  available  (Sanders  et.  al.,  2002). 

In-Place  After  Action  Review  (AAR). 

Immediately  after  the  completion  of  each  trial  run,  a  five  to  ten  minute  structured 
interview  occurred  called  the  In-Place  AAR.  This  interview  was  administered  by  an  ARI 
researcher  with  each  of  the  player  personnel  still  at  their  designated  workstations  in  the  mock-up 
C2  Vehicle,  as  shown  in  Figure  2.  This  setting  allowed  players  to  review  and  refer  to  their 
tactical  displays  as  they  provided  a  summary  of  their  performance  during  the  trial  run.  This 
setting  also  supported  video  and  audio  recording  of  all  player  comments  and  references  to  their 
tactical  displays.  This  brief  interview  with  each  player  asked  for  a  short  recapitulation  of  “what 
went  right  and  what  went  wrong”  relative  to  their  duty  position  during  the  run.  Results  below 
are  limited  to  a  sample  comment  from  each  command  group  participant. 

•  Run  8,  Battlespace  Manager  Comment:  “This  run  represents  the  most  effective 
integration  of  ground  maneuver  with  aerial  reconnaissance  so  far.  The  targets  we  initially 
missed,  we  were  able  to  acquire  eventually  and  to  engage.” 

•  Run  9,  Information  Manager  Comment:  “Right  now  the  workload  is  OK  at  this  station. 

As  we  get  more  detailed  in  doing  the  targeting  and  engagement,  that  may  change  in 
future  runs.” 

•  Run  5,  Effects  Manager  Comment:  “I  got  pretty  hurried  doing  Inter-netted  Unattended 
Ground  Sensors  (lUGS)  fields  and  Loiter  and  Attack  Munition  (LAM)  engagements  and 
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Precision  Attack  Munition  (PAM)  engagements  simultaneously,  and  then  trying  to 
(redirect  my  attention)  up  north  to  look  at  stuff.” 


Figure  2.  In-Place  AAR  in  the  Commander’s  Support  Environment. 

•  Run  8,  Unit  Cell  Commander  Comment:  “In  this  run  there  was  better  integration  and 
synchronization  of  assets,  however,  our  prep  reconnaissance-by-fire  technique  using 
LAMs  and  the  smart  munitions  had  minimal  effect. 

After  Exercise  Survey 

This  short  survey  was  administered  immediately  after  the  command  group  participants  exited 
the  C2  Vehicle.  The  After  Exercise  Survey  contained  seven  items  that  asked  the  command 
group  players  to  assess  key  research  issues  (e.g.,  “What  CSE  features  require  more  automation, 
and  why?”).  Sample  player  responses  concerning  CSE  automation  requirements  ineluded: 

•  Robotic  vehicles  should  move  autonomously  to  designated  targets  and  locations 

•  There  is  no  back  up  for  the  loss  of  aerial  reconnaissance. 

•  Need  easier  and  more  flexible  re-tasking. 

•  Need  some  sort  of  automated  battle  damage  assessment  (BDA). 

•  Improve  planning  collaboration  among  the  command  group’s  workstations. 

The  After  Exercise  Survey  also  asked  players  to  rate  their  perceived  workload  and 
performance  success  (i.e.,  “How  successful  were  you  in  accomplishing  what  you  needed  to 
do?”).  Summary  results  on  successful  performance  as  a  function  of  trial  complexity  are 
provided  in  Figure  3.  Across  the  three  subordinate  members  of  the  command  group,  there  is  a 
notable 

decrease  in  estimates  of  success  at  the  Too  High  level.  In  contrast,  the  Commander’s  ratings  are 
relatively  low  and  constant  across  trials,  perhaps  indicating  higher  performance  standards. 
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Ratings  of  Performance  Success  as  a  Function  of  Complexity 


Complexity 

Figure  3.  Ratings  of  performance  success  by  trial  complexity. 

Exit  Interview 

On  the  final  day,  after  all  trials  were  completed,  a  structured  Soldier-Machine-Fimctions 
interview  was  conducted  with  the  Unit  Cell  Command  and  all  three  Battle  Staff  Managers.  This 
interview  investigated  the  human  functions  thought  to  be  required  for  successful  Unit  Cell  task 
performance.  A  small  sample  of  comments  provided  by  the  players  for  several  key  functions 
follows: 

•  Maintain  Battle  Rhythm  Comments:  “There  is  a  need  for  automated  sensor-ganging; 
don’t  make  us  do  this  ourselves.”  “BDA  needs  to  be  automated,  hands-off,  with 
‘iconology’  like  a  slash  over  a  targeted  icon  to  indicate  it  has  been  engaged.” 

•  Use  Planning  Tools  Comments:  “Need  a  White  Board.  We  probably  don’t  need  more 
Intel,  what  we  need  is  a  better  way  to  share  analysis.”  “We  need  a  3D  tool  for  a  detailed 
review  of  terrain  to  learn  how  to  better  use  it;  identify  potential  enemy  keyhole 
positions.” 

•  Maintain  Situational  Awareness  Comments:  “The  present  task  allocation  is  good.” 
“When  we  have  a  collaborative  White-Board  we  can  develop  our  own  fires.” 

Human  Functions  Assessment 

A  post  hoc  analysis  of  the  taped  verbal  communications  among  command  group  participants 
and  higher  echelons  was  conducted  to  assess  human  functions.  For  the  nine  Experiment  1  runs, 
commxmications  were  transcribed  and  separated  into  analytic  chunks  or  blocks  of 
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communication  b^ed  on  a  Human  Functions  Rating  Scale  developed  for  that  purpose,  and  the 
time  duration  for  each  block  was  recorded.  Analysts  used  this  scale  to  rate  each  block  of 
communication  with  respect  to  Source,  Function  and  Factor. 

•  Source:  Within  Cell,  Cell  to  Black,  Cell  to  White,  and  Black  to  White. 

•  Function:  Plan,  Move,  See,  and  Shoot. 

•  Factor:  Mission,  Enemy,  Terrain,  Troops,  Time,  and  Civilians  (METT-TC). 

For  clarity.  Source  categorizations  were:  Within  Cell  corresponds  to  communications  within  the 
command  group;  Cell  to  Black,  communications  between  command  group  and  Black  Cell 
headquarters;  Cell  to  White,  communications  between  command  group  and  White  Cell 
headquarters;  and  Black  to  White,  communications  between  headquarter  cells. 

Figure  4  summarizes  how  communications  were  distributed  on  the  average  by  Source  and 
Factor,  across  the  nine  runs.  Approximately  40%  of  Within  Cell  communications  addressed 

07^^  ^  command  group  communications  split  between  Mission 

(27 /o)  and  Enemy  (28 /o)  factors.  In  contrast,  headquarter  communications  between  Black- 

^^te  were  predominantly  about  Enemy  forces  (77%),  with  the  remainder  devoted  to  the 
Mission  Factor  (23%). 


Percentage  of  Verbal  Communication  by 
Factor  for  each  Source 


Within  Cell  CelFBIack  CelFWhite 

Source 


Black- White 


^  Mission 
S  Enemy 
^Terrain 
E3  T  roops 
□  Time 
S  Civilians 


Figure  4.  Percentage  of  verbal  communication  by  Factor  and  Source. 

Mid-Scale  Environment  Refinement  and  Research  Issues 

n  u  Experiment  1  was  a  decisive  first  step  in  the  empirical  assessment  of  Future 

Combat  Systems  (FCS)  concepts  of  command  and  control  by  a  Unit  Cell  over  a  large  number  of 
m^ed  and  robotic  entities.  The  Experiment  1  efforts  to  build  a  transformation  environment  for 
FCS  C  were  in  many  ways  successful.  However,  as  expected,  substantial  work  remains  in 
refining  and  sustaining  this  environment  for  future  research  efforts.  First,  this  section  provides  a 
bnef  overview  of  some  general  lessons  learned  for  refining  this  environment.  Second  more 
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specific  lessons  learned  about  human-system  integration  and  the  research  issues  of  Allocation, 
Autonomy,  Authority,  and  Awareness  are  considered. 

Lessons  Learned  Overview 

As  part  of  its  advisory  role  to  the  FCS  C  Program,  ARI  provided  the  Program  Manager  a 
brief  After  Action  Review  (AAR)  on  Experiment  1 .  This  review  identified  three  Sustain  and 
Improve  topics  to  refine  this  transformation  environment.  Sustain  topics,  at  least,  partially 
addressed  during  Experiment  1  are  bulleted  below: 

•  Proactive  and  exploratory  research  compatible  with  FCS  transformation  objectives. 

•  Deliberate  practice  to  promote  proficiency  for  new  command  and  control  skills. 

•  Functional  analysis  focused  on  complementing  human  performance. 

Each  of  these  Sustain  topics  is  discussed  elsewhere  in  the  current  section  Mid-Scale 
Transformation  Environments,  and  that  discussion  is  not  repeated  here.  Improve  topics  not 
adequately  addressed  during  Experiment  1  are  discussed  below. 

Improve  Training.  New  system  training  is  always  a  challenge.  The  introduction  of  a  new 
technology  creates  new  tasks  and  roles  for  people  at  every  level  and  introduces  new  errors, 
strategies,  and  expectations.  The  training  challenge  is  compounded  by  a  prototype  technology, 
such  as  the  CSE,  as  it  includes  the  difficulty  of  developing  a  training  program  for  an 
undocumented  and  continuously  changing  suite  of  complex  functions.  Shortcomings  in  training 
for  Experiment  1  were  notable,  therefore,  but  to  some  degree  unavoidable.  Training 
improvements  for  subsequent  experiments,  however,  must  benefit  from  more  complete 
documentation  on  system  functionality  at  both  individual  and  collective  task  levels.  Training 
must  also  address  the  FCS  C2  program’s  focus  on  more  innovative  and  collaborative  command 
and  control.  The  operational  concepts  embodied  by  the  CSE  require  new  and  unprecedented 
levels  of  collaboration  within  the  Unit  Cell,  across  echelons,  and  between  humans  and  machines. 
Training  for  future  efforts  should  include  practical  exercises  that  target  each  of  these  levels  of 
collaboration,  and  that  are  based  on  clear  training  objectives.  In  summary,  training 
improvements  should: 

•  Document  system  functionality  at  individual  and  collective  task  levels. 

•  Develop  practical  exercises  for  collaboration  within  the  Unit  Cell,  across  echelons,  and 

between  humans  and  machines. 

Improve  the  Link  Between  Thinking  and  Doing.  Despite  current  rhetoric  that  the  Unit 
Cell  and  Team  levels  au’e  primarily  doers  not  planners,  the  CSE  should  be  modified  to  provide 
horizontal  and  vertical  integration  for  planning  activities.  For  example,  during  Experiment  1  the 
players  had  to  repeatedly  hand-jam  intelligence  updates  from  higher  echelons  into  their  CSE 
workstations  just  prior  to  execution.  Another  example,  Unit  Cell  players  and  0/Cs  strongly 
stressed  the  requirement  for  a  Whiteboard,  or  John  Madden  Chalkboard,  type  capability.  The 
ARI  notes,  however,  that  such  boards  usually  support  planning,  rehearsal,  and  review  modes,  but 
not  execution.  Typically,  after  the  commander  depicts  a  course  of  action  on  a  Whiteboard, 
players  must  manually  re-input  the  same  planning  information  into  different  formats  or 
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applications  required  for  execution.  In  sum,  CSE  improvements  should  provide  direct  links 
between  thought  and  action,  command  and  control,  across  manned  and  unmanned  systems  This 
linkage  should  decrease  command  and  control  workload,  error,  and  response  times  In 
summary,  thought  and  action  linkages  should: 

•  Ensure  horizontal  and  vertical  integration  for  planning  activities. 

•  Develop  a  Whiteboard  capability  that  transforms  plans  into  actions. 

Improve  Enabling  Technologies.  A  primary  focus  of  the  PCS  C2  effort  is  to  design  and 
develop  a  prototype  technology  called  the  CSE  that  efficiently  and  effectively  meets  a  Unit 
Cell  s  future  co^and  and  control  requirements.  Although  the  CSE  capabilities  available  to  the 
p  ayers  dunng  Experiment  1  were  impressive,  extensive  refinement  of  the  CSE  is  required  to 
provide  and  shape  conmand  and  control  enabling  technologies.  Low-level,  more  basic  enablers 
include  a  wide  range  of  human  computer  interface  improvements.  Selected  examples  include- 

cos”Z  S%’ ""  “R^-tasking  problems 

cost  me  SA  —Battle  Space  Manager).  Mid-level  enablers  are  needed  to  ensure  needed 

command  and  control  information  is  proactively  provided  across  planning  and  execution  modes 

Selected  examples  include:  on-the-move  depictions  of  critical  “see”  and  “be  seen”  information ' 

such  as  sensor  footpnnts  and  platform  mter-visibility;  intuitive  and  effective  alert  mechanisms-  ’ 

and  terrain  analysis.  Particularly,  high-level  enablers  are  needed  to  make  current  Unit  Cell  ’ 

cojKep^  viable  FCS  even  a.  Uie  Uni.  Cell  level,  is  a  net-dependent  system 

Selected  examples  include:  netted  sensors  for  cross-cueing  and  verification,  and  for 

locations;  netted  fires  for  sensor-shooter  links  and  fire  coordination-  and 

netted  communications  for  force  multiplication  within  the  Unit  Cell  and  across  echelons  ^d  for 

should-""^^^^"^^"^  single-point  failures).  In  summary,  enabling  technologies 


•  Provide  low-level,  more  basic,  human  computer  interface  improvements 

•  Provide  mid-level  proactive  “push”  of  information  in  planning  and  execution 

•  Provide  high-level  integration  to  include  netted  sensors,  fires,  and  communications. 

Lessons  Learned  on  Research  Issues 

Observations  made  during  Experiment  1 ,  contributed  to  identification  of  the  research 
issues  of  Allocation,  Autonomy,  Authority,  and  Awareness.  Lessons  learned  concerning  thL 
research  issues  is  summarized  in  Appendix  A.  The  lessons  documented  in  Appendix  A  focus 
pnmarily  on  improving  research  methods  to  address  these  issues  in  future  FCS  C2  experiments. 

As  noted  in  Appendix  A,  only  the  issue  of  how  to  allocate  functions  and  tasks  across  the 
co^and  group  and  supporting  technology  was  clearly  targeted  before  and  during  Experiment  1 
Lessons  documented  in  Appendix  A  reflect  findings  from  Experiment  1,  projections  about  fumre 
experiments  and  the  previously  reviewed  literature  on  past  efforts  at  improving  the  integration  of 
umans  and  machines^  The  incremental  development  phases  required  to  build  a  transformation 
enviromnent,  such  as  FCS  C2,  greatly  influences  when  and  how  research  issues  can  and  may  be 
addressed.  For  example  functions  and  tasks  that  burdened  the  command  group  in  ExperimL  1 
may  be  shed,  or  shared  by  new  technologies  in  future  experiments. 
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In  closing,  the  evolving  FCS  C  mid-scale  transformation  environment  should  become  an 
important  contributor  in  the  exploration  of  command  and  control  concepts  to  obtain  viable 
solutions.  Building  on  lessons  learned,  this  environment  should  inereasingly  reflect  many  of  the 
key  features  that  characterize  a  transformation  environment,  particularly  the  collaborative  and 
iterative  efforts  of  interdependent  Operational,  Technical,  and  Human  Performance  teams.  By 
design,  the  mid-scale  nature  of  the  FCS  environment  ably  complements  some  shortcomings 
inherent  to  large-scale  transformation  environments.  However,  even  a  mid-scale  research 
approach  can’t  adequately  address  many  key  research  requirements  best  approached  in  small- 
scale  transformation  environments. 

Small-Scale  Transformation  Environments 

This  section  focuses  on  the  need  for,  and  potential  of,  small-scale  transformation 
environments,  particularly  in  support  of  FCS  command  and  control.  The  complementary  role  of 
small-scale  transformation  environments  in  relation  to  larger  environments  is  first  considered. 
Next,  a  description  of  a  small-scale  transformation  environment  currently  under  development  by 
ARI  at  Fort  Knox  stresses  how  technology  expands  the  potential  of  a  small-scale  environment, 
including  integration  with  larger  environments. 

Value  Added 

This  report  stresses  that  a  complementary  range  of  transformation  environments  are 
needed  to  achieve  the  profound  changes  required  for  FCS  force  transformation.  A  similar 
conclusion  was  reached  for  transformation  across  the  services,  and  for  transformations  directed 
at  command  and  control  (Gold,  1999;  Taylor,  et.  al.,  1996).  A  more  unique  conclusion  made 
here  is  that  the  driving  and  integrative  focus  across  transformation  environments  and  efforts 
should  be  the  need  to  shape  or  transform  technology  to  complement  human  performance. 

The  decisive  value  added  by  small-scale  transformation  environments  is  the  ability  to 
maintain  a  human-centric  focus.  Failure  to  maintain  that  focus  results  in  technology  that  adds 
complexity,  difficulty,  frustration,  and  failure  per  se.  Notable  examples  of  such  failures  in  the 
area  of  command  and  control  were  previously  considered  under  Research  Issues.  Human 
considerations  are  often  lost  in  large-scale  efforts,  including  modem  warfare,  where  humans 
labor  to  support  a  maze  of  complex  machines  not  designed  to  support  them  (Cordesman  & 
Wagner,  1996). 

What  are  the  roles  of  small-scale  transformation  environment?  How  are  those  roles 
unique  from  the  research  conducted  in  larger  environments?  How  do  those  roles  complement, 
integrate  with,  the  research  conducted  in  larger  environments?  Two  primary  roles  for  smaller- 
scale  transformation  environments  are  bulleted  and  discussed  below. 

•  Breeding  Ground — to  cull  and  refine  innovations  for  larger  environments. 

•  Proving  Ground — ^to  assess  and  resolve  key  issues  from  larger  environments. 

As  preface  to  that  discussion,  complementary  roles  of  small-  to  large-  scale  FCS 
transformation  environments  are  depicted  in  Figure  5.  The  environments  depicted  in  Figure  5 
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Figure  5.  Integrating  FCS  Transformation  Environments. 

m  only  example  envijo^ents  most  closely  related  to  ARI's  small-scale  environment  at  Fort 
^ox,  namely  *e  FCS  C  program  discnssed  in  the  previous  section  and  the  Unit  of  Action 
Mmeuver  Battle  Lab  at  Fort  toox  discussed  later  in  this  section.  There  are  multiple  examples 
of  FCS  transfonnation  at  each  level  of  effort  that  should  be  integrated.  In  particulm  larae-Se 
exiles  should  include  the  Annys  fonnal  FCS  development  L  a^uisiS  effSs.  ffg^5 
SO  attempts  to  depict  how  ^mmand  and  control  provides  a  common  and  shared  framework  for 
integratmg  transformation  efforts,  particularly  in  the  form  of  advanced  systems  and  displays. 

Breeding  Ground 

A  primary  role  of  a  small-scale  transformation  environment  is  to  serve  as  a  breeding 
ffom  of  innovative  concepts  for  larger  environments.  Smaller  environments  should  serve  as 
eadmg  edge  providers  of  potentially  useful  human-machine  solutions  to  larger  environments 
As  a  provider  of  innovation,  the  small-scale  environment  must  be  able  to  cull  or  eliminate  for 
transfer  inadequate  solutions,  and  prepare  or  refine  innovations  to  a  point  adequate  for  transfer. 

To  perform  Ae  role  of  a  breeding  ground,  a  small-scale  transformation  environment 
should  possess  all  of  the  charactenstics  previously  identified:  empirical,  scalable,  iterative 
collaborative,  and  human-centered.  An  innovation  often  begins  as  a  high-risk  approximation  of 
^potentially  useful  solution  for  many  settings  that  is,  in  fact,  not  ready  for  any  setting  Before 

^stly  and  complex  large-scale  efforts,  risk  and  readiness  should  be 
addressed^  Smaller-scale  efforts  can  empirically  identify  and  iteratively  refine  innovations  until 
adequate  for  transfer  to  larger  environments. 

Proving  Ground 

cr.  A  Pri^aiy  role  of  a  small-scale  transformation  environment  is  to  serve  as  a  proving 

ground  to  help  resolve  key  issues  identified  in  larger  environments.  Small  environments  permit 
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more  detailed  and  systematic  assessment  of  the  human-system  integration  issues  that  invariably 
surface  during  lager  scale  efforts,  including  operational  tests  on  actual  equipment.  Testing  and 
analysis  may  resolve  the  issue  or  refine  a  solution  to  a  point  ready  for  transfer  back  to  the  larger 
environment.  Refinement  may  include  human-centered  requirements  from  users,  or  technical 
requirements  to  integrate  the  subject  innovation  with  other  technology  in  the  larger  environment. 

Again,  performing  the  role  of  a  proving  ground  requires  that  a  small-scale  transformation 
environment  have  all  of  the  characteristics  of  a  transformation  environment  previously  identified. 
Based  on  lessons  learned,  including  the  human-system  integration  literature  reviewed,  the  issues 
that  emerge  in  larger  scale  environments  often  result  fi'om  a  failure  to  address  human-centric 
concerns  early  on.  The  ability  of  small-scale  transformation  environment  to  resolve  human¬ 
centric  design  deficiencies  is  crucial  to  force  transformation. 

ARI’s  Emerging  Small-Scale  Transformation  Environment 

As  part  of  ARI’s  research  on  PCS  concept  exploration  and  training  our  team  is 
developing  a  multi-purpose  research  laboratory  at  Fort  Knox.  For  concept  exploration  and 
development  in  support  of  command  and  control,  this  facility  is  being  developed  as  a  small-scale 
transformation  environment.  For  future  training,  the  vast  changes  envisioned — including 
embedded  and  distributed  treiining,  performance  support  systems,  and  intelligent  tutors — ^also 
require  an  environment  for  exploration  and  transformation.  Description  of  this  emerging 
environment,  therefore,  focuses  on  its  potential  for  addressing  human-system  integration  and 
training  issues. 

Situated  Performance 

The  theme  of  situated  performance  is  introduced  here  to  convey  the  power  and  potential 
of  a  small-scale  transformation  environment.  This  discussion  of  situated  performance  focuses  on 
the  validity  and  value  of  work  performed  in  a  smaller  environment,  and  the  relation  of  that  work 
to  and  from  larger  transformation  environments. 

Performance  and  the  readiness  to  perform  depend  on  the  situation.  Situational  emphasis 
is  reinforced  by  the  Army’s  tradition  of  performance-based  training  defined  by  tasks,  conditions, 
and  standards.  This  emphasis  is  reinforced  by  recent  empirical  findings  that  performance  is 
rarely  based  on  isolated  cognition,  but  rather  on  an  adaptive  cycle  of  perception  and  action  in 
response  to  the  situation  (Kirlik  &  Bisantz,  1999).  Behavior  out  of  context,  including  traditional 
small-scale  laboratory  research,  is  often  not  predictive  of  real  world  behavior. 

With  our  growing  reliance  on  technology,  including  computer-mediated  work,  the 
situation  as  experienced  is  increasingly  a  digital  representation.  Consider  the  integral  role  that 
digital  representations  on  (9  systems  will  play  for  FCS  command  and  control.  By  design, 
systems  situate  performance,  and  are  readily  coupled  to  military  simulations  that  generate 
realistic  operational  situations.  The  ability  of  a  modem  small-scale  transformation  environment 
to  couple  systems  and  simulations  to  more  fully  situate  human  performance  provides  atypical 
power  and  potential.  This  ability  expands  the  situational  scope  of  small-scale  transformation 


21 


environments  to  literally  global  proportions,  commensurate  with  the  reach  of  simulation, 
systems,  and  FCS  mission  requirements. 

For  users,  digital  technology  is  the  keystone  of  FCS  transformation,  particularly  for 
command  and  control.  This  is  underscored  by  the  FCS  concept  of  a  network-centric  force  with 
netted  communication,  sensor,  and  fire  capabilities.  The  common  operational  picture  depicted 
on  C  systems  situates  user  perceptions  and  cognitions.  The  C^  system  also  situates  performance 
by  serving  as  a  micro\vorld  system  in  which  inputs  and  outputs  literally  correspond  to  behaviors 
in  the  actual  situation,  including  the  behaviors  of  autonomous  systems. 

For  researchers  and  developers,  digital  technology  situates  their  performance  by 
providing  unprecedented  presence  in  the  world  of  users.  This  presence  affords  researchers  and 
developers  the  ability  to  directly  share,  control,  and  track  the  situations  that  ground  users’ 
perceptions,  cognitions  and  actions. 

The  ability  to  share  directly  and  concurrently  the  situation  and  conditions  affecting  user 
performance  helps  researchers  and  developers  relate  performance  to  context,  to  the  purpose  of 
performance.  The  ability  to  modify  and  control  performance  situations — including  repeatable 
and  re-configurable  conditions,  and  automatic  and  immediate  feedback— enables  systematic  and 
Iterative  assessment  of  issues  and  refinement  of  innovations  for  larger  environments.  The  ability 
to  digitally  track  and  collect  performance  data  makes  it  possible  for  researchers  and  developers 
to  precisely  correlate  task  conditions,  performance,  and  measures  of  performance. 

Core  Assets 

,  ..  emerging  status  of  ARI’s  small-scale  transformation  environment,  this  section 

descnbes  this  environment’s  core  assets  currently  available  and  envisioned.  These  assets  are 
summarized  under  technical,  operational  and  human  performance  dimensions. 


Technical.  Overall,  the  technical  infrastructure  for  this  small-scale  transformation 
environment  provides  an  empirical  venue  that  affords  users,  researchers  and  developers  the 
ability  to  customize  tasks  and  conditions  in  order  to  iteratively  explore  and  transform  concepts 
into  viable,  human-centric  solutions.  There  are  two  core  technical  assets  emphasized  for  this 
small-scale  transformation  environment: 

•  Situate  users  in  realistic  situations  through  simulation. 

•  Empower  users  to  act  on  realistic  situations  through  microworld  systems. 

For  simulation,  our  team  s  emphasis  is  on  virtual  simulation  to  ensure  user-in-the-loop 
performance  and  feedback.  The  principal  form  of  virtual  simulation  for  this  environment  will  be 
Semi-Automated  Forces  (OneSAF),  the  Army’s  mainstream  virtual  simulation  for  concept 
exploration,  training  and  acquisition.  Currently,  the  most  recent  version  called  OneSAF  Testbed 
Baseline  (0TB)  is  operating  in  our  facility.  As  OneSAF  evolves  it  should  continue  to  excel  at 
situating  user  performance  in  realistic  and  futuristic  operational  situations. 
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The  ability  of  any  simulation  to  represent  future  operations  and  situations  is  a  distinct  and 
recurrent  challenge.  Ongoing  work  on  PCS  concept  exploration  by  the  Unit  of  Action  Maneuver 
Battle  Lab  (UAMBL)’  at  Fort  Knox  continues  to  drive  a  cycle  of  OneSAF  software  upgrades 
that  reflect  the  Army’s  transformation  from  concepts  to  FCS  solutions.  The  OneSAF  distribution 
agreement  between  UAMBL  and  others,  including  ARI,  ensures  continued  access  to  OneSAF 
upgrades  as  new  versions  are  developed.  In  addition,  the  distributed  and  open-ended  nature  of 
OneSAF  supports  linkages  between  virtual,  constructive  and  live  simulation  environments. 

Of  special  interest,  OneSAF  directly  supports  the  requirement  to  integrate  and 
complement  FCS  efforts  across  transformation  environments.  The  re-configurable  nature  of 
OneSAF  enables  a  range  of  environmental  scales,  including  the  relatively  large-scale 
transformation  efforts  conducted  by  the  UAMBL,  and  the  mid-scale  transformation  efforts 
described  for  the  FCS  C^  program.  As  a  common  medium  across  environments,  OneSAF-based 
research  will  greatly  facilitate  the  ability  of  our  small-scale  transformation  environment  to  serve 
as  a  breeding  and  proving  ground  for  larger  environments. 

Other  forms  of  simulation  will  supplement  our  team’s  efforts  to  customize  tasks  and 
conditions  to  research  issues  and  objectives.  Currently,  the  Mission  Planning  and  Rehearsal 
Suite  (MPARS)  is  installed  to  help  researchers  plan,  design,  and  develop  simulation  exercises  as 
well  as  supporting  materials  such  as  maps  and  overlays.  A  completed  MPARS  file  can  be  used 
to  initialize  a  variety  of  simulation  applications,  including  OneSAF.  In  addition,  Microsoft 
Visual  Basic  provides  a  relatively  low  cost  and  highly  exportable  medium  for  customizing  tasks 
and  conditions,  to  include  construction  of  a  scaled-world  (e.g.,  Ehret  et  al.,  2000)  and/or 
synthetic  task  environment  (e.g.,  Schiflett  &  Elliott,  2000). 

For  digital  C^  systems,  the  core  asset  required  is  the  ability  to  provide  micro  world 
representations  of  users’  operational  situations.  Currently,  the  Force  XXI  Battle  Command, 
Brigade  and  Below  (FBCB^)  system  is  installed  in  our  facility.  The  FBCB^  system  is  fielded  and 
serves  as  the  primary  Of  system  for  digitally  linking  brigade  and  battalion  echelons  to  soldier  and 
platform  levels.  A  futuristic  C^  prototype  developed  by  the  UAMBL  for  their  work  on  concept 
exploration  is  also  installed.  This  prototype  serves  as  an  “objective”  C^  system  exemplifying 
many  of  the  command  and  control  capabilities  anticipated  for  FCS.  Other  advantages  of  this  Of 
prototype  are  its  compatibility  with  OneSAF,  its  relatively  unique  status  as  a  fully  instrumented 
C^  system,  and  its  direct  linkages  to  UAMBL  efforts.  Another  Cf  system,  currently  under 
investigation  by  our  team,  is  the  previously  described  CSE  prototype  being  developed  by  the 
FCS  C^  program  to  support  a  Unit  Cell’s  command  group. 

Instrumentation  to  support  data  capture  and  analysis  is  another  important  technical  asset 
for  this  environment.  Initial  assets  required  for  post  hoc  analysis  of  the  Experiment  1  results 
from  the  FCS  program  were  minimal,  the  ability  to  playback  video  and  audio  recordings  as 
well  as  the  ability  to  analyze  results  with  resident  statistical  packages.  Assets  now  include  the 
ability  to  record  audio  and  video  across  multiple  in-house  research  settings  in  analog  and  digital 


'  The  UAMBL  expands  and  reshapes  what  was  formerly  the  Mounted  Maneuver  Battle  Lab  (MMBL). 
Microsoft  Visual  Basic  is  a  registered  trademark  of  Microsoft  Corporation. 


23 


format,  and  to  relay  any  these  recordings  of  user  performance  to  an  observer/controller  room. 
Instrumentation  will  also  provide  the  ability  to  capture  all  user-computer  interactions  precisely 
correlated  with  the  situation  in  which  they  occur,  including  user-C^  system  interactions  during 
simulated  operations. 

Operational.  The  user  is  essential  to  the  value  and  validity  of  the  research  conducted  in  a 
transformation  environment.  Moreover,  the  overall  contribution  by  operational  personnel  is 
much  more  than  serving  as  research  participants.  Recall,  members  of  the  PCS  Operational 

Team  collaborated  on  C  system  design,  developed  mission  requirements  and  scenarios,  and 
provided  the  expertise  and  vision  of  Friendly  and  Enemy  commanders  as  well  as  an  0/C  team  for 
exercise  control  and  AARs.  Products  and  guidance  from  this  Operational  Team  will  be 
leveraged  in  developing  a  small-scale  transformation  environment.  In  particular,  this  includes 
lessons  learned  and  issues  identified  in  this  larger  effort  that  might  be  better  addressed  in  our 
team’s  smaller  environment. 

The  ARI  Field  Unit  at  Fort  Knox  is  positioned  to  ensure  user  involvement  in  nearly  all 
aspects  of  the  research  planned  for  our  small-scale  transformation  environment.  Particularly 
Fort  I^ox’s  Commanding  General  is  the  proponent  for  FCS  and  the  Unit  of  Action,  the  echelon 

proponent,  the  UAMBL  will  expand  and  reshape  to  serve 
as  the  FCS  integration  center,  the  hub  of  future  force  developmental  work. 

ARI  has  a  long  and  sustained  history  of  support  to  Fort  Knox’s  Battle  Lab  that  includes 
f  ^  (e.g.,  Leibrecht,  Meade,  Schmidt,  Doherty,  &  Lickteig 

1994)  and  on  the  Battle  Lab’s  Battle  Command  Concept  Experimentation  Program  (e.g.  Throne 
Holden,  &  Lickteig,  2000).  Our  Unit  will  continue  work  in  this  larger-scale  transformation 
environment  and  this  will  include  importing  innovations  into  the  UAMBL,  and  exporting  key 

research  issues  from  the  Battle  Lab  into  our  small-scale  environment  for  assessment  and 
resolution. 

In  addition,  ARI  continues  to  work  with  the  Armor  School  and  the  University  of 
Mounted  Warfare’s  efforts  to  tram  the  officers,  non  commissioned  officers,  and  enlisted  soldiers 
of  the  future  force.  This  includes  ongoing  work  for  the  Armor  Captains'  Career  Course  formerly 
known  as  the  Armor  Officers’  Advanced  Course  (AOAC).  Two  aspects  of  this  work  are  research 
on  distance  le^ing,  particularly  for  the  Reserve  Component,  and  training  methods  to  promote 

Captains  Career  Course  at  Fort  Knox  has  integrated 
a  Think  Like  A  Commander  (TLAC)  instructional  module  with  cognitive  drills  to  promote  more 
adaptive  battle  command  thinking  (U.S.  Army  Research  Institute,  2001). 

Our  Unit’s  association  with  the  Armor  School  should  provide  us  an  excellent  pool  of 
research  participants  for  user-in-the-loop  assessment  and  feedback.  The  proximity  of  our  Unit 
and  the  Armor  School  should  support  more  informal  arrangements  for  formative  evaluations,  and 
troop  support  requirements  for  more  rigorous  assessments.  Moreover,  the  current 
and  future  students  of  the  Armor  School  embody  the  personal  background  and  intuitive  vision 
required  to  explore  and  exploit  technology,  and  to  ensure  technology  is  shaped  to  complement 
human  performance. 
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Human  Performance.  The  core  human  performance  asset  for  this  transformation 
environment  is  ARI’s  FBC  team  at  Fort  Knox,  the  authors  of  this  report.  The  members  of  this 
team  are  educated  in  the  behavioral  and  cognitive  sciences  and  experienced  in  the  application  of 
science  to  Army  issues,  including  human-system  integration.  The  FBC  team  will  assist  the 
Army’s  transformation  efforts  through  this  small-scale  environment.  Our  Field  Unit  will 
continue  its  work  in  larger  environments,  including  the  FCS  program  and  the  UAMBL. 

Additional  assets  within  ARI  include  our  Unit’s  collaboration  with  other  Research  Units, 
particularly  in  support  of  a  Science  &  Technology  Objective  (STO)  '"'‘Methods  and  Measures  of 
Commander-Centric  Training.'"  Collaborative  assets  outside  of  ARI  include  our  ongoing  work 
with  DARPA  and  CECOM  on  the  FCS  C^  program,  as  well  as  numerous  other  government 
research  and  development  organizations  working  on  Army  transformation. 

Contractual  efforts  funded  by  ARI’s  Research  Unit  at  Fort  Knox  provide  additional 
human  performance  assets.  Two  contracts  directly  related  to  this  small-scale  transformation 
environment  will  address  the  need  to  scale-down  research  approaches  by  means  of  a  synthetic 
task  environment,  and  the  need  to  develop  prototype  performance  support  systems.  The  research 
objectives  in  developing  a  synthetic  task  environment  for  selected  C^  functions  include  the 
ability  to  explore  and  develop  future  force  concepts,  and  to  train  future  leaders  in  the  d5mamics 
of  emerging  concepts. 

Research  objectives  in  developing  a  prototype  performance  support  system  include  the 
ability  to  provide  just-as-needed  job  support,  in  a  manner  readily  adaptable  to  changes  in  human 
and  machine  performance  requirements.  The  pervasive  presence  and  potential  of  technology 
blurs  the  distinction  between  training  before  performing  versus  supporting  while  performing. 
Notably,  performance  support  systems  reflect  the  theme  of  situated  performance. 

Finally,  two  Small  Business  Innovation  Research  (SBIR)  efforts  are  briefly  noted  that 
will  contribute  to  the  human  performance,  and  technical,  assets  of  our  small-scale  transformation 
environment.  One  SBIR  is  directed  at  developing  a  prototype  Intelligent  Tutoring  System  (ITS) 
for  conceptual  thinking,  namely  battle  command  reasoning.  The  objectives  for  this  effort 
include:  automated,  on-line  tutoring  with  deliberate  practice  opportunities,  and  a  partial  answer 
to  the  growing  problem  of  limited  expert  human  tutors.  This  challenging  effort  attempts  to 
transfer  the  TLAC  training  approach  from  human  to  automated  tutors,  and  provide  the  adaptive 
thinking  skills  increasingly  required  for  command  and  control. 

The  second  SBIR  effort  is  directed  at  the  design  and  development  of  intelligent 
technologies  called  “interface  agents”  to  assist  in  the  command  and  control  of  automated 
systems,  such  as  robotic  and  intelligent  entities.  Research  objectives  for  this  effort  include  the 
development  of  tasking,  monitoring  and  coordinating  agents  that  can  translate,  and  even 
anticipate,  human  inputs  for  lower-level  automated  systems.  The  expected  benefit  is  to  reduce 
human-machine  interactions  and  training  requirements,  and  support  decision-making  and 
communication  across  and  within  echelons. 

In  closing,  as  this  small-scale  research  approach  is  still  emerging,  no  research  findings  or 
lessons  learned  from  this  environment  are  provided.  However,  two  examples  are  provided  that 
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represent  the  research  anticipated  in  this  environment.  First,  our  previously  reported  post  hoc 
Ma  ysis  of  human-machine  functions  for  the  FCS  program  was  the  first  research  performed  in 
Ais  environment.  This  work,  addressing  the  issue  of  Allocation  and  continuing  throughout  the 
FCS  C  program,  exemplifies  this  environment’s  role  as  a  proving  ground  for  larger  scale 
transformation  environments. 

Second,  our  SBIR  effort  to  develop  interface  agents  to  assist  in  the  command  and  control 
of  automyed  systems  began  only  recently.  Anticipated  deliverables  from  this  effort  are  software 
modules  d;;;ectly  com 

Vu"  and  other  FCS  transformation  efforts.  This  work,  addressing  Autonomy 

and  Authority  issues  and  providing  innovations  ready  for  transfer  to  UAMBL,  exemplifies  this 
environment’s  role  as  breeding  ground  for  larger  scale  transformation  environments. 

SUMMARY  AND  CONCLUSIONS 

*  began  by  considering  the  human-system  integration  research  requirement  to 

meet  F^  objectives,  with  a  special  focus  on  command  and  control  at  the  small  unit  level.  This 
research  requirement  is  based  on  the  unprecedented  alliance  of  humans  and  machines  posed  by 
CS.  In  support  of  that  requirement,  research  issues  and  approaches  were  identified  that  reflect 
ongoing  work  by  the  U.S.  Army  Research  Institute’s  Future  Battlefield  Conditions  team  at  Fort 

l^OX. 


Fom  overarching  research  issues  for  improving  human-system  integration  in  command 
and  contro  were  selectively  identified  based  on  their  perceived  relevance  to  future  command  and 
control.  Allocation,  Autonomy,  Authority,  and  Awareness.  A  review  of  human-system 
integration  literature  distilled  two  primary  conclusions.  First,  an  unintended  consequence  of 
technology  is  an  increased  burden  on  humans,  as  human-system  integration  is  often  an 
unattended  issue.  Second,  the  burden  on  humans  from  advances  in  military  technology  is  caused 
not  just  by  technology,  but  also  by  inflated  expectations  about  technology. 

As  research  context  for  the  research  approaches  identified,  key  themes  related  to  force 
transformation  were  established.  First,  the  profound  changes  essential  to  transformation  require 
a  comprehensive  rese^ch  approach  that  integrates  and  complements  efforts  across  a  range  of 
small  to  large  research  environments.  Second,  the  driving  and  integrative  focus  across 
transformation  efforts  for  FCS  should  be  the  need  to  shape  or  transform  technology  to 
complement  human  performance. 

The  notion  of  a  transformation  environment  was  introduced  to  overcome  shortcomings  in 
l^ge  and  small  transformation  efforts  cobbled  together  in  a  manner  that  does  not  attend  to 
hum^  perfoimance  issues.  A  working  definition  of  a  transformation  environment  was  provided 
and  Its  key  characteristics  identified:  empirical,  scalable,  iterative,  collaborative,  and  human- 
centered.  The  notion  of  a  leam-by-doing  environment  was  reinforced  in  subsequent  descriptions 
of  mid-  and  small-scale  transformation  environments.  ^ 

A  case  example  of  an  emerging  mid-scale  transformation  environment  was  provided 
based  on  the  FCS  C  program.  The  stated  purpose  of  the  FCS  C'  program  is  to  test  the 
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hypothesis  that  digitization  of  current  battlefield  operating  systems  enables  a  new  approach  to 
command  and  control.  Description  of  this  environment  highlighted  how  it  reflects  the  key 
features  of  a  transformation  environment,  particularly  the  resources  and  products  of  three 
interdependent  Operational,  Technical,  and  Human  Performance  Teams. 

Selected  results  from  the  PCS  program’s  Experiment  1  emphasized  human-system 
integration  findings,  based  on  ARI’s  involvement  in  this  research  effort.  These  results  focused 
on  the  PCS  requirement  that  a  small  command  group  at  the  Unit  Cell  level  can  command  and 
control  an  expansive  mix  of  manned  and  autonomous  systems.  More  general  sustain  and 
improve  lessons  learned  for  refining  this  environment  were  presented.  And,  more  specific 
lessons  learned  about  human-system  integration  and  the  issues  of  Allocation,  Autonomy, 
Authority,  and  Awareness  for  the  Unit  Cell  command  group  were  examined. 

A  case  example  of  an  emerging  small-scale  transformation  environment  was  provided 
that  reflects  the  efforts  of  ARI  in  support  of  PCS  concept  exploration  and  future  force  training. 
The  report  concluded  that  a  decisive  value  added  by  small-scale  transformation  environments  is 
the  ability  to  maintain  a  human-centric  focus,  often  lost  in  large-scale  efforts.  Two  unique  and 
complementary  roles  for  small-scale  transformation  environments  were  identified  as:  a  breeding 
ground  to  cull  and  refine  innovations  for  transfer  to  larger  environments;  and,  a  proving  ground 
to  assess  and  resolve  key  issues  from  larger  environments. 

The  theme  of  situated  performance  underscored  the  fact  that  human  performance  is 
almost  always  based  on  an  adaptive  cycle  of  perception  and  action  in  response  to  the  situation. 
Behavior  out  of  context,  a  typical  concern  with  small-scale  research  environments,  often  fails  to 
predict  real  world  behavior.  However,  technology  now  provides  the  ability  to  simulate  realistic 
performance  situations,  and  the  ability  to  emulate  and  act  on  real  world  situations  with  digital 
systems.  Small-scale  transformation  environments  developed  to  exploit  these  technical 
capabilities  possess  atypical  power  and  potential. 

finally,  a  status  report  described  the  core  assets  for  ARI’s  emerging  small-scale 
transformation  environment.  These  assets  will  provide  an  empirical  venue  that  affords  users, 
researchers  and  developers  the  ability  to  customize  tasks  and  conditions  in  order  to  iteratively 
explore  and  transform  concepts  into  viable,  human-centric  solutions.  Core  assets  were 
summarized  imder  technical,  operational  and  human  performance  dimensions. 

Technical  assets  currently  installed  are  designed  to  situate  users  in  real  world  situations 
through  simulation,  such  as  OneS  AP,  and  empower  users  to  act  on  the  same  situations  through 
microworld  systems,  including  fielded  and  prototype  systems.  Technical  assets  also  include  a 
range  of  instrumentation  to  support  data  capture  and  analysis,  including  the  ability  to  capture  all 
user-computer  interactions  precisely  correlated  with  the  situation  in  which  they  occur. 

Operational  assets  are  essential  to  the  value  and  validity  of  research  in  a  transformation 
environment,  and  extend  far  beyond  serving  as  research  participants.  Operational  personnel  will 
develop  mission  requirements  and  scenarios,  collaborate  on  system  design,  and  provide  the 
expertise  and  vision  required  to  explore  and  develop  PCS  concepts.  Our  unit’s  proximity  to  and 
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support  of  the  proponent  for  PCS,  the  UAMBL,  and  the  Armor  School  will  provide  invaluable 
access  to  operational  assets. 

Human  performance  assets  for  this  transformation  environment  center  on  ARI  personnel 
at  Fort  Knox  educated  in  the  cognitive  and  psychological  sciences  and  experienced  in  the 
application  of  science  to  Army  issues.  Our  ongoing  work  in  larger  environments,  including  the 
PCS  C  program  and  the  UAMBL,  will  relate  and  integrate  the  work  performed  in  our  small- 
scale  environment  to  mdfrom  larger  transformation  environments. 

In  closing,  a  reminder  that  the  primary  purpose  of  this  report  is  to  disseminate  and 
coordinate  ARI  s  efforts  within  the  body  of  research  on  human-system  integration,  and 
particularly  PCS  research.  The  intended  audience  includes  members  of  the  user,  researcher  and 
developer  community  who  might  benefit  from,  or  provide  benefit  to,  the  Army’s  ongoing  PCS 
research  program.  f  & 
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Appendix  A 

Lessons  Learned  on  Research  Issues 

Observations  made  during  Experiment  1,  contributed  to  identification  of  the  research 
issues  of  Allocation,  Autonomy,  Authority,  and  Awareness.  However,  only  the  issue  of  how  to 
allocate  fimctions  and  tasks  across  the  command  group  and  supporting  technology  was  clearly 
targeted  before  and  during  Experiment  1.  Lessons  learned,  therefore,  are  based  on  findings  from 
Experiment  1,  projections  about  future  experiments,  and  the  previously  reviewed  literature  on 
past  efforts  at  improving  the  integration  of  humans  and  machines. 

In  preface,  the  incremental  development  required  to  build  a  transformation  environment, 
such  as  PCS  C2,  greatly  influences  when  and  how  research  issues  can  be  addressed.  Limited 
capabilities  and  requirements  in  Experiment  1  clearly  limit  findings  and  lessons  learned.  For 
example,  functions  and  tasks  that  burdened  the  command  group  in  Experiment  1  may  be  shed,  or 
shared  by  new  technologies  in  future  experiments.  These  lessons  learned  focus  primarily  on  the 
research  methods  required  to  address  these  issues  in  future  PCS  C2  experiments. 

Allocation 

Despite  our  team’s  limited  success  at  developing  methods  for  assessing  human-machine 
allocation  of  PCS  command  and  control  functions  and  tasks,  a  basic  conclusion  is  that  substantial 
improvements  in  research  methods  are  needed.  For  subjective  assessments  provided  by  the 
command  group,  functional  categories  on  questionnaire  and  interview  methods  will  be  revised. 
For  future  experiments,  these  data  collection  instruments  may  ask  respondents  to  assess 
functionality  based  on  more  basic  and  familiar  categorizations,  such  as  the  Plan,  See,  Move, 
Strike  and  Sustain  categories  anticipated  for  a  Unit  Cell,  and  METT-TC.  For  objective 
assessment  of  command  group  performance,  needed  method  refinements  are  bulleted  below  and 
then  briefly  discussed. 

•  Refine  the  categorization  and  coding  of  human-machine  functionality,  including  verbal, 
nonverbal  and  cognitive  behaviors. 

•  Refine  the  ability  to  visually  review  human-computer  interactions  in  recorded  runs  and 
exercises. 

•  Develop  the  ability  to  automatically  measure  human-computer  interactions,  particularly 
command  group  and  CSE  interactions. 

•  Identify  the  training  requirements  and  skill  levels  needed  for  FCS  command  and  control, 
particularly  at  the  Unit  Cell  level. 

Refinements  are  needed  to,  and  beyond,  the  Human  Functions  Rating  Scale  codes 
developed  to  analyze  verbal  communications  from  Experiment  1.  An  adequate  analysis  of  FCS 
command  and  control  must  address  both  human  and  machine  functions  and  how  these  are 
related.  Moreover,  the  analysis  of  human  functions  must  also  include  nonverbal  behaviors, 
including  perception  and  cognition.  Methods  are  needed  not  only  to  capture  and  collate  a  wide 
range  of  human  behaviors,  but  also  to  better  understand  behavioral  components  by  assessing 
behavioral  patterns  across  components,  or  relating  behaviors.  For  example,  humans  routinely 
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analyze  verbalizations  by  relating  the  communication  to  perceptual  cues  about  what  the  speaker 
IS  looking  at,  pointing  to,  or  doing  at  that  time.  ^ 

A  first  method  refinement  needed  is  to  improve  the  codes  used  to  assess  human  functions 
in  Experiment  1 .  Method  changes  will  focus  on  more  precise  categorizations  of  the  Unit  Cell 
players  behavior  including  verbalizations,  to  improve  inter-rater  reliabilities.  Breaking  the 
verbalization  blocks  into  smaller  units  before  coding  should  help  us  reach  inter-rater  agreement 
minimum  of  80  A  related  step  is  to  extend  and  refine  the  analysis  codes  developed  for 

player  verbalizations  in  a  manner  that  maps  or  matches  CSE  functionality  and  other  player 

behaviors.  In  sum,  a  common  coding  strategy  across  human  and  machine  functions  is  needed  to 
assess  human-system  integration. 

refinement  needed  is  the  ability  to  visually  review  the  human-computer 
interactions  of  the  comm^d  group  on  recorded  videotape.  During  Experiment  1,  the  low  quality 

?  f  f f  the  identification  of  specific  player  actions  with^ 

the  CSE  interface.  As  a  result,  the  functional  analysis  was  largely  limited  to  the  recorded 

rrfJnHft  experiments,  digital  recording  formats  should  allow  analysts  to  visually 

fimctiMs  interactions  with  the  CSE  interface  as  they  perform  operational  tasks  and 

A  third  method  requirement  is  the  ability  to  automatically  measure  player  and  CSE 
mteractions  to  supplement  the  manual  analysis  of  human  and  machine  interactions.  Manual 
analysis  of  audio  and  video  recordings  is  laborious,  time-consuming,  and  prone  to  error  In 
contrast,  automated  measures  of  human-computer  interaction  should  increase  the  scope  and 
precision  of  the  fimctional  analysis,  and  significantly  decrease  the  analytic  workload  A  key 
recommendation  for  improving  functional  analysis  in  future  experiments  is  to  instrument  the 
ob  in  a  manner  that  provides  a  log  of  all  human-computer  interactions. 

Iev=k  W  ?r<!  the  training  requirements  and  skill 

levels  tor  bCS  command  ^d  control,  particularly  at  the  Unit  Cell  level.  The  PCS  C2 

expenmentation  program  is  exploring  new  paradigms  in  command  and  control.  These 

para  igms,  particularly  expectations  about  multi-functional  roles  that  encompass  and  extend 

traditional  command  ^d  control  functions,  will  entail  new  training  and  skill  requirements  By 

identifying  and  describing  command  group  behaviors  for  the  Unit  Cell,  the  analysis  of  human- 

Autonomy  and  Authority 

Autonomy  and  Authority  were  not  predefined  research  issues  for  Experiment  1 ,  as  noted 
imitations  m  automation  during  the  first  experiment,  nevertheless,  posed  relatively  few  threats 
to  the  command  group  s  efforts  to  command  and  control  the  Unit  Cell.  However  ongoing 
refinements  to  the  CSE  including  a  suite  of  alerts,  advisories  and  decision  aiding  routines  are 
expected  to  raise  concerns  about  Autonomy  and  Authority  in  future  experiments. 
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Findings  from  Experiment  1  do  provide  some  interesting  indicators  about  the  relatively 
high  degree  of  automation  the  command  group  is  expecting.  In  particular,  the  command  group 
expressed  the  need  for  more  autonomous  capabilities  across  maneuver  and  sensor  platforms  to 
reduce  their  workload,  and  to  maintain  the  desired  pace  of  operations  or  battle  rhythm. 

Concerning  sensors,  for  example,  the  command  group  stressed  the  requirement  that 
unmanned  aerial  vehicles  (UAVs)  have  an  auto-evade  capability.  After  detecting  jamming 
signals,  auto-evade  should  allow  UAVs  to  automatically  alter  their  routes  in  order  to  avoid  air 
defense  artillery.  They  also  stated  that  sensors  should  communicate  with  one  another  to 
collaboratively  resolve  target  detection,  recognition  and  identification  requirements.  The 
command  group  also  felt  overburdened  by  battle  damage  assessment  requirements,  and  incapable 
of  adequately  performing  such  assessments  given  the  Experiment  1  sensor  capabilities.  They 
recommended  that  micro  UAVs  should  automatically  key  off  Unit  Cell  target  engagements,  then 
fly  out  to  the  target  and  confirm  kills,  and  report  their  assessment  to  the  command  group’s  CSE 
workstations. 

Two  method  requirements,  in  particular,  are  stressed  to  assess  the  issues  of  Autonomy 
and  Authority  in  future  experiments.  First,  data  collection  methods  should  focus  on  the 
observability  of  the  automation  to  the  command  group.  The  extent  to  which  the  command  group 
can  observe,  query  and  understand  automated  fvmctions  in  real  time  is  critical  to  ensuring  that  the 
command  group  is  actually  in  command  and  in  control.  This  includes  automation  embedded  in 
Unit  Cell  platforms,  such  as  robotic  scout  vehicles  and  UAVs,  and  automation  in  their  CSE 
workstations. 

Second,  the  ability  to  automatically  measure  human-system  interactions  is  needed  to 
assess  the  command  group’s  use  of  and  response  to  automation.  A  key  method  requirement  for 
assessing  the  issues  of  Autonomy  and  Authority,  as  noted,  is  to  instrument  the  command  group’s 
interactions  with  CSE,  particularly  in  response  to  alerts,  advisories  and  decision  aiding  provided 
by  the  CSE. 

Issues  concerning  management-by-exception  and  -consent  were  also  not  addressed 
during  Experiment  1 .  However,  given  the  relatively  high  workload  experienced  by  the  command 
group  during  Experiment  1  and  their  requests  for  more  automation,  a  reasonable  prediction  is 
that  they  may  opt  for  a  management-by-exception  strategy  as  more  automation  is  introduced. 
However,  optimal  management  strategies  are  an  empirical  question  highly  dependent  on  the 
overall  suite  of  automated  capabilities  ultimately  developed  for  this  transformation  environment. 

Awareness 

Awareness  was,  at  least  indirectly,  addressed  by  Experiment  1  ’s  focus  on  the  ability  of 
the  command  group  and  Unit  Cell  to  “See.”  Many  objective  measures  of  performance  related  to 
detecting  and  identifying  Enemy  forces,  as  well  as  limiting  the  Enemy’s  ability  to  detect  and 
identify  Friendly  forces  were  collected.  As  noted,  one  such  measure  was  the  percentage  of  threat 
elements  located  by  the  Unit  Cell  during  a  trial.  Results  on  these  measures  were  documented, 
but  restricted  to  the  Interim  Report  for  Experiment  1  available  from  the  Program  Manager  (PM) 
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Recall,  thiS' report’s  prior  discussion  of  the  Awareness  issue  focused  particularly  on  how 
to  maintain  human  awareness  of  “the  big  picture”  as  technology  advances.  The  ability  to 
maintain  overall  awareness  amidst  the  deluge  of  new  information  provided  by  technology  was 
not  directly  addressed  during  Experiment  1.  However,  observations  during  Experiment  1 
indicated  that  the  command  group,  and  particularly  the  commander,  lost  the  big  picture  at  critical 


The  problem  of  maintaining  overall  awareness  became  a  central  point  of  discussion 
dunng  several  of  the  AARs  that  followed  the  Experiment  1  trial  runs.  The  critical  times 
identified  m  the  AARs  usually  occurred  during  periods  of  intense  activity,  such  as  concentrated 
engagements.  At  such  times,  all  four  members  of  the  command  group  seemed  to  simultaneously 
focus  on  the  same  small  area  of  the  battlefield  where  the  engagements  were  occurring. 


Data  supporting  a  loss  of  awareness  is  often  indirect  or  subjective.  Notably,  replays  of 
each  run  during  an  AAR  included  the  visible  views  of  the  battlefield  as  displayed  on  the  CSE 
workstations  for  each  member  of  the  command  group.  One  indirect  indicator  available  from 
taese  replays  was  when  the  members  of  the  command  group  had  zoomed  in  their  tactical 
isplays  to  more  directly  view  the  engagement  area  in  question.  At  such  times,  relatively  large 
^eas  of  the  overall  battlefield  situation  were  not  visible  on  any  command  group  member’s  ^ 
display.  Dunng  the  actual  runs,  there  were  also  notable  lags  in  the  command  group’s  detection 
of  Enemy  activity  m  battlefield  areas  not  currently  visible  on  any  member’s  tactical  display. 


Other  indicators  the  command  group  had  temporarily  lost  the  big  picture  included  their 
reactions  and  co^ents  during  the  run  and  subsequent  AARs.  Command  group  verbalizations 
during  the  r^  often  indicated  surprise  when  undetected  Enemy  activity  was  belatedly  noticed  by 
a  member  of  the  command  group,  or  reported  to  them  by  headquarters.  Such  detections  were 
usuaiy  followed  by  zooming  displays  out  to  view  a  larger  area.  Comments  during  the  AAR 
included  admissions  by  the  command  group  that  they  had  “lost  the  bubble”  and  their  discussions 
about  needing  a  standing  operating  procedure  (SOP)  to  ensure  at  least  one  member  of  the 
command  group  always  monitored  the  overall  tactical  area. 


I  f  experiments,  several  methods  are  recommended  for  assessing  Awareness  or 

loss  of  the  big  picture.  Digital  recording  formats  should  allow  analysts  to  visually  identify  ’ 
human-machine  interactions,  including  zooms  and  visible  map  areas,  indicating  that  the  overall 
battlefield  was  not  being  monitored.  Visual  analysis  supplemented  by  verbal  analysis  of 
command  group  comments  might  better  indicate  surprise  or  lags  in  detecting  key  events. 

Another  needed  method  for  assessing  Awareness  is  the  ability  to  automatically  measure 
human  and  machine  interactions,  as  noted  earlier.  For  example,  prior  work  by  ARI  on 
instrumented  C  systems  developed  a  variety  of  automated  measures  to  assess  human-machine 
performance  (Throne,  Holden  &  Lickteig,  2000).  One  of  those  measures,  called  Map  Area 
automatically  compared  the  battlefield  areas  visible  on  the  displays  of  a  battalion  level 
command  and  staff  organization.  A  Map  Area  measure  for  the  FCS  program  would  help 
analysts  identify  patterns  in  display  use  that  might  indicate  problems  in  maintaining  Awareness. 
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More  standard  methods  for  assessing  situational  awareness  (S  A)  may  also  be  needed. 
Two  methods  in  particular  are  recommended  from  the  rather  large  toolkit  of  measures  currently 
available  (Endsley  &  Garland,  2000).  First,  a  direct  measurement  approach  called  Situation 
Awareness  Global  Assessment  Technique  (SAG AT)  is  widely  considered  the  best  SA  measure 
available.  Second,  a  less  intrusive  method  also  recommended  is  the  use  of  experimental  probes 
to  obtain  “testable  responses”  of  SA.  Probes  are  a  classic  form  of  experimental  manipulation 
that  when  carefully  crafted  can  help  relate  SA  to  purposeful  performance. 


